+33 (0)6 85 22 07 25 mario.diaznava@st.com

Edge AI use cases Demonstrators

In ANDANTE, more than 19 demonstrators over 14 use cases in 5 applications domains were implemented by the partners based on the methods, tools, building blocks, ASICs and platforms developed in ANDANTE as well as reference platforms like the NVIDIA Jetson Nano to validate and evaluate the project results. Furthermore, for the realization of these prototypes, the partners gathered data, developed hardware components like new sensors and trained and optimized AI algorithms. During the final review of ANDANTE in February 2024, the outcome of this work was presented providing an overview of the setups, used technologies, obtained evaluation results, impact of the work and the lessons learned. In the following we provide more details on the specific domains. However, please note that the platform and ASIC numbers refer to the descriptions provided in the “Advanced Neuromorphic Processors for IoT Edge Solutions” section of this newsletter.

Digital Industry

Digital Industry includes the two use cases, “Indoor Positioning, Recognition and People Counting” and “Color Classification at the Edge for Quality Control”. For the first one, three different setups were realized. Setups 1 and 2 were based on project platform 4.2 and SpiNNAKER2 (a 152 PEs digital neuromorphic chip) respectively and were used to explore topics around SNNs like parallelization of SNNs. The last setup leveraged ASIC 3.1b to evaluate the developed FeFET memory. The advantages of each setup were also discussed in terms of accuracy, real-time sensing, and power consumption. Furthermore, the color classification use case was implemented with ASIC 3.2 and board 4.1 to provide environment for investigating ANN concepts for extreme edge applications.

People Recognition and Counting

Digital Farming

Digital farming includes the two use cases “Autonomous Weeding System for Crop and Weed Detection” and “Tomato Pest and Disease Prediction”. Both applications seek Edge AI components to reduce the use of chemical substances (herbicides and pesticides), low energy consumption and costs to better protect the environment, provide healthy food and ensure greater sustainability. These use cases require high accuracy and very low power consumption to increase the device and system autonomy. Thus, the corresponding partners evaluated the neuromorphic technologies of ASIC 2.1 and 4.1a platforms which provide sufficient accuracy with lower power consumption than NVIDIA Jetson Xavier.

Crops detection

Transport and Smart Mobility

The domain of mobility includes five use cases: “Drones for Border Surveillance”, “Underwater Drone for Acoustic Signal Classification”, “3D Objects Detection and classification of Road Users based on LiDAR and Camera”, “Drones for Robust Autonomous Landing”, and “Path Planning for Autonomous Steering”. All of them require high accuracy, to be energy efficient, real-time processing and a small factor. To achieve these goals the first four use cases investigated the usage of ASIC 2.1 and platform 4.1a for AI acceleration as well as other approaches based on FPGAs. Moreover, the second use case explored ASIC 1.3, ASIC 2.1 and corresponding platforms to determine the potential of SNN and digital ANN acceleration for underwater drones. Last but least, the fifth use case was leveraged to push forward the development of the GrAI Matter Labs hardware. The results show that ANDANTE’s ASIC solution outperforms FPGA and reference platforms in terms of power consumption and real-time processing which are key requirements to for example increase the autonomy and payload of drones. Furthermore, the use of neuromorphic chips in the perception pipeline for autonomous driving could be of great interest and benefit in the field of transportation and mobility.

3D Object Detection and Camera Semantic Segmentation

Healthcare

Healthcare includes three use cases: “Multimodal Image Processing and Device Tracking”, “Ultrasound Acquisition and Processing” and “Glucose Monitoring.” For the first two use cases, the SENeCA accelerator (FPGA-based) was explored for safer, faster, efficient, and precise medical procedures. For the first use case, low latency is essential and for the second, accuracy. The results show that these two requirements were not met by the SENeCa accelerator and that an ASIC implementation could be a more adequate solution to achieve this demanding performance. Furthermore, the last use case investigated SNNs via platform 4.2 for accurate, energy efficient, and non-invasive monitoring devices.

Image based device tracking based on CNN model

Detect Covid infection through lung ultrasound

Digital Society

Two use cases were considered in this domain: “Neuromorphic Technology for Auditory Processing in Consumer Devices” and “Neuromorphic Technology for Visual-based Human-Computer Interaction”. The first use case is divided into four sub-use cases. Three of them leverage ASIC 1.3 to prove the potential of SynSense SNN technology for audio processing at the edge and the last one was used to evaluate ASIC 3.1 and platform 4.3 for voice activity detection. The use case results show extremely low power consumption, essential for continuous sound monitoring in, for example, home and factory environments.

Continuous monitoring and classification of the ambient audio, for consumer headsets
or hearing aids. Four-class detection of the audio scene. Low-latency, low-power detection

Furthermore, the second use case regarding human-computer interaction explored SNNs for this field via the Speck DevKit of SynSense for detecting user attention on a device.

Conclusions

Overall, at the end of the final review of the entire ANDANTE project, the consortium has concluded the following points based on the work around the use cases and their evaluation:

  • ANDANTE has produced many great results which have set the stage for further developments after the project and ensuring Europe’s competitiveness.
  • Neuromorphic technologies are maturing and slowly reaching the productivity plateau of the Gartner hype cycle, leading to a clearer picture of what is possible with them across ANDANTE’s different application areas.

 

CONSORTIUM

ANDANTE’s activities are based on collaborative research between large and small industrial companies, leading research centers and universities from seven European countries.