The Embedded Learning and Sensing Systems (ELSS) group is a young team of researchers working on the development of a new generation of AI-powered embedded systems that operate on resource-constrained hardware in a predictable and trustworthy manner. As the number of sensors integrated into IoT devices grows, so is the amount of data they produce, the need to locally process this data, and the resulting complexity of the engineered systems. Modern computational models are increasingly based on deep learning principles. These models exert severe demands on local resources, need to adapt to the target environment, and ensure robustness to unexpected changes of the local context. We believe in "less is more" and "small can be mighty" when it comes to running deep models on resource-constrained devices.
Our group works on design principles, optimization methods and tools necessary to run AI-based models on embedded and mobile devices. We cover research questions along the pipeline spanning smart sensor systems, high-quality embedded sensing and information processing, pruning and adaptation methods for deep models to match resource constraints of IoT devices. We care about energy consumption, environmental sustainability and data privacy, which is reflected in applications of our work and deployed prototypes of AI-powered embedded systems.
Exemplary research challenges tackled within the group include:
- How do deep model training and weight pruning techniques impact individual classes?
- What is the best way to store and process sparse tensors on embedded devices?
- Can we improve robustness of sparse deep models to natural distribution shifts?
- How to locally adapt a model to distribution shifts in a target environment?
- What are specefic sparsity patterns that are worth to be supported in hardware?
- How to preserve privacy in time series sensor data? How to bound temporal privacy leakage?
- Can we enable low-power sensing through smart measurement prediction and scheduling?
- How to improve convergence speed and bandwidth usage of distributed model training?
- How to make most value from sensor measurements within a fixed energy budget?
- How to improve environmental footprint of training deep models?
We publish our research results at top conferences in Machine Learning (e.g., ICLR, NeurIPS, ICML, CVPR) and Cyber-Physical Systems (IPSN, SenSys, EWSN). We always give our master students interesting and relevant research questions and encourage them (plus provide all the help needed) to prepare, publish and present their first papers at conferences or workshops. We are very proud that our students even managed to receive best paper awards for their work!
Time window: November 2022 - March 2025
Type: FFG-funded project in cooperation with Pro2Future, University of St. Gallen (HSG)
Description: AI-based methods such as deep learning require resources for training and operation. Safety‐critical functionality may be not be suited for cloud computing, as the risk of intermittent connectivity and communication latencies are an issue. To get solutions closer to embedded devices, the product itself must exhibit cognitive abilities by utilizing embedded intelligence. To achieve this vision, E‐MINDS investigates novel training and deployment methods that foster the development, training, testing, and deployment of AI/ML models on resource‐constrained embedded devices, to achieve dependable embedded intelligence for cognitive products and production systems.
Time window: December 2023 - November 2025
Type: FFG-funded KIRAS project in cooperation with FH JOANNEUM, other institutes of TU Graz and industrial partners
Description: Water supply and wastewater treatment plants are steadily transforming from traditional physical infrastructures to
cyber-physical systems (CPS). Digitalization brings new vulnerabilities and attack surfaces for attacks from cyberspace. There has been an increase in reported cyberattacks on water management assets, demonstrating that working preventive measures are as necessary as early detection and location of attacked system components. Traditional mechanisms for detecting cyberattacks are becoming increasingly ineffective. Our contribution to the project is to develop advanced and robust AI-based tools for early attack detection and better situational and risk assessment.
Time window: May 2022 - April 2026
Type: FWF-funded project in cooperation with FH JOANNEUM and other institutes of TU Graz
Description: Electronics-based systems (EBS) are becoming more and more prevalent in production, infrastructure and transport, but are only accepted if people trust these systems. The researchers in the doctoral programme Dependable ElectroNIc-Based SystEms (DENISE) will explore concepts, methods and application-oriented tools to make EBS more reliable. The project deepens the very good relationship between FH Joanneum and Graz University of Technology through a joint doctoral programme. The aim of our team in the project is to make AI-powered EBS more trustworthy. This is particularly challenging when a model is trained or refined on an embedded device using locally and autonomously acquired data with little or no supervision.
Time window: April 2021 - March 2024
Type: FFG-funded project in cooperation with Pro2Future, AVL and JKU
Description: The project aims to develop an infrastructure for cognitive (IoT-based and AI-based) vehicle fleet monitoring, which includes collection, evaluation, interpretation and use of vehicle data in the context of various data‐driven services. The project will cover several services and use cases for data‐driven support and expansion of the development processes as well as the provision of services on the vehicle (e.g., predictive maintenance). The fleet data can be utilized to detect new trends in mobility. Our goal is to develop new methods to achieve fast data collection in automotive applications, design and development of novel and efficient algorithms and AI-based data processing methods for automotive environments. (1) We will enable rapid prototyping of onboard measurement for fast data collection and method development, (2) modular device design to capture use‐case specific data and enable future extensions (sensors and services), and (3) capture, interpret and preprocess data on the device for responsive local services.
Time window: November 2018 - present
Description: Deep learning methods are increasingly used to solve complex tasks, yet little is known about the choice of the best training data, architecture, and model capacity to match available hardware and energy resources. The trade-offs and optimization potential in this space are not well understood, and are being explored in this project. We also aim to develop deep learning models that show reliable generalization not only for conventional in-distribution but also for out-of-distribution scenarios. We focus on resource constrained IoT devices and their autonomous operation on the edge, where the data is scarce and environmental dynamics is non-stationary. To achieve the goal we study the loss landscape of deep neural networks and try to disentangle the impact of model’s architecture, overparameterization, training data, augmentation, training objectives, hyperparameters, regularization, etc. on generalization. We also look at optimized models operating in resource-constrained environments from two perspectives: (1) their generalization and robustness and how to improve these, and (2) how to make compressed models run efficiently on specialized hardware.
Time window: September 2012 - present
Type: This research was supported by multiple funding agencies over the years including Nano-Tera.ch, FFG, TU Graz, Pro2Future (LocSense project), best paper award budgets, successful collaborations with domain experts and data providers
Description: Low-cost environmental sensors and environmental models present interesting use-cases for testing, optimizing and improving machine learning models. For example, low-cost gas sensors may drift over time or their measurements may be affected by other environmental processes and environmental dynamics. Sophisticated machine learning models for accurate sensor calibration can help to compensate for these effects. On-device sensor data processing, however, may face severe resource constraints including processing power, memory and energy budget, that need to be taken into account. Another example is that some types of chemical gas sensors are power-hungry and machine learning can be used to replace actual measurements with high-quality predictions. Finally, sensor data coming from IoT sensors measuring gases and particle concentrations in the ambient air help to push the limits of today's air quality maps by extending the conventional networks of static high-quality measurement stations with dense IoT measurements. The challenge is to show that it is possible to build high-quality and high-resolution air quality maps using low-cost, less precise, low-resolution, less stable, noisy sensors. While successfully solving this challenge, we managed to improve the accuracy of air quality models by accounting for air pollution transfer, and used our models to understand the impact of COVID-19 lockdown measures on local air quality.
Recent publications (see full list here):