ITI Teams/Embedded Information Processing/Research

Embedded Information Processing Group

The Embedded Information Processing group is a young team of researchers working on the development of a new generation of AI-powered embedded systems that operate on resource-constrained hardware in a predictable and trustworthy manner. As the number of sensors integrated into IoT devices grows, so is the amount of data they produce, the need to locally process this data, and the resulting complexity of the engineered systems. Modern computational models are increasingly based on deep learning principles. These models exert severe demands on local resources, need to adapt to the target environment, and ensure robustness to unexpected changes of the local context. We believe in "less is more" and "small can be mighty" when it comes to running deep models on resource-constrained devices.

Our group works on design principles, optimization methods and tools necessary to run AI-based models on embedded and mobile devices. We cover research questions along the pipeline spanning smart sensor systems, high-quality embedded sensing and information processing, pruning and adaptation methods for deep models to match resource constraints of IoT devices.  We care about energy consumption, environmental sustainability and data privacy, which is reflected in applications of our work and deployed prototypes of AI-powered embedded systems.

Exemplary research challenges tackled within the group include:
- How do deep model training and weight pruning techniques impact individual classes?
- What is the best way to store and process sparse tensors on embedded devices?
- Can we improve robustness of sparse deep models to natural distribution shifts?
- How to locally adapt a model to distribution shifts in a target environment?
- What are specefic sparsity patterns that are worth to be supported in hardware?
- How to preserve privacy in time series sensor data? How to bound temporal privacy leakage?
- Can we enable low-power sensing through smart measurement prediction and scheduling?
- How to improve convergence speed and bandwidth usage of distributed model training?
- How to make most value from sensor measurements within a fixed energy budget?
- How to improve environmental footprint of training deep models?
- Can we improve resolution and accuracy of air pollution maps?

We publish our research results at top conferences in Machine Learning (e.g., ICLR, NeurIPS, ICML, CVPR) and Cyber-Physical Systems (IPSN, SenSys, EWSN). We always give our master students interesting and relevant research questions and encourage them (plus provide all the help needed) to prepare, publish and present their first papers at conferences or workshops. We are very proud that our students even managed to receive best paper awards for their work!


Third-party funded research projects

DENISE - Doctoral School for Dependable Electronic-Based Systems

Time window: May 2022 - April 2026
Type: FWF-funded project in cooperation with FH JOANNEUM and other institutes of TU Graz

Description: Electronics-based systems (EBS) are becoming more and more prevalent in production, infrastructure and transport, but are only accepted if people trust these systems. Reliability is therefore becoming the cornerstone for the social acceptance of electronics-based systems. The researchers in the doctoral programme Dependable ElectroNIc-Based SystEms (DENISE) will explore concepts, methods and application-oriented tools to make EBS more reliable. The project deepens the very good relationship between FH Joanneum and Graz University of Technology through a joint doctoral programme. DENISE creates an integrated research framework across disciplinary boundaries and links reliability concepts of sensors with AI-powered embedded systems and networked embedded devices. The aim of our team in the project is to make AI-powered EBS more trustworthy. This is particularly challenging when a model is trained or refined on an embedded device using locally and autonomously acquired data with little or no supervision. This project will define methodologies in three key areas that lay the foundation for a safe and trusted AI-powered EBS: bias, explainability, and safety.

Involved researchers:


CORVETTE: Cognitive Sensing Framework for Vehicle-Fleet Driven Data Services

Time window: April 2021 - March 2024
Type: FFG-funded project in cooperation with Pro2Future, AVL and JKU

Description: The project aims to develop an infrastructure for cognitive (IoT-based and AI-based) vehicle fleet monitoring, which includes collection, evaluation, interpretation and use of vehicle data in the context of various data‐driven services. The project will cover several services and use cases for data‐driven support and expansion of the development processes as well as the provision of services on the vehicle (e.g., predictive maintenance). The fleet data can be utilized to detect new trends in mobility. Our goal is to develop new methods to achieve fast data collection in automotive applications, design and development of novel and efficient algorithms and AI-based data processing methods for automotive environments. (1) We will enable rapid prototyping of onboard measurement for fast data collection and method development, (2) modular device design to capture use‐case specific data and enable future extensions (sensors and services), and (3) capture, interpret and preprocess data on the device for responsive local services.

Involved researchers:


D4Dairy: Digitalisation, Data integration, Detection and Decision Support in Dairying

Time window: October 2018 - Sept 2022
Type: FFG COMET-Project is supported by BMVIT, BMDW and the provinces of Lower Austria and Vienna

Description: The overall goal of D4Dairy is the generation of added value for herd management as well as the improvement of animal health, animal welfare and product quality by creating a well-developed data network and by exploiting the opportunities offered by new digital AI-based technologies and analytical methods. The specific objectives of D4Dairy are (a) to capture enormous amounts of diverse data available on the farms, and from project partners along the milk chain; (b) to aggregate this data into a central database, ensure compliance with legal requirements and develop a concept of interoperability; (c) to perform advanced analyses in order to detect risk factors and identify early predictors of cow health problems using machine learning methods,  to detect and gather information about the impact of housing climate on animal health and welfare; (d) to develop data-based strategies to reduce the use of antimicrobials and implement quality assurance programs, and (e) to provide the information obtained from the analyses for decision support using innovative tools that are easy to apply, operate in real-time in an automated fashion. The task of our team in this project is to enable privacy-preserving data integration and sensor data quality assurance.

Involved researchers:


Internal strategic projects

Resource-efficient and Data-efficient Deep Learning

Time window: November 2018 - present

Description: Deep learning methods are increasingly used to solve complex tasks, yet little is known about the choice of the best training data, architecture, and model capacity to match available hardware and energy resources. The trade-offs and optimization potential in this space are not well understood, and are being explored in this project. We also aim to develop deep learning models that show reliable generalization not only for conventional in-distribution but also for out-of-distribution scenarios. We focus on resource constrained IoT devices and their autonomous operation on the edge, where the data is scarce and environmental dynamics is non-stationary. To achieve the goal we study the loss landscape of deep neural networks and try to disentangle the impact of model’s architecture, overparameterization, training data, augmentation, training objectives, hyperparameters, regularization, etc. on generalization.  We also look at optimized models operating in resource-constrained environments from two perspectives: (1) their generalization and robustness and how to improve these, and (2) how to make compressed models run efficiently on specialized hardware.

Selected publications:

Involved researchers:


Privacy Preserving Techniques in Sensor Data for Good

Time window: November 2020 - present

Description: IoT sensor data is often collected in very private areas of human life and it is possible to derive sensitive information from it. For example, daily routine of a person or occupancy status of a flat can easily be predicted when looking at smart meter readings; wearable devices reveal the stress level of the person wearing it. However, many sensor data related applications would greatly benefit if the collected data could be shared or used to train machine learning models. Privacy preserving machine learning is an active field of research, but so far IoT sensor data did not receive the necessary attention. In our work we try to bridge the gap between useful but private data and its applications, by researching methods on how to share and processes this data without sharing or leaking private information.

Selected publications:

Involved researchers:


Adaptive Low-power Sensing and Accurate Prediction for Clean Air

Time window: September 2012 - present
Type: This research was supported by multiple funding agencies over the years including Nano-Tera.ch, FFG, TU Graz, Pro2Future (LocSense project), best paper award budgets, successful collaborations with domain experts and data providers

Description: Low-cost environmental sensors and environmental models present interesting use-cases for testing, optimizing and improving machine learning models. For example, low-cost gas sensors may drift over time or their measurements may be affected by other environmental processes and environmental dynamics. Sophisticated machine learning models for accurate sensor calibration can help to compensate for these effects. On-device sensor data processing, however, may face severe resource constraints including processing power, memory and energy budget, that need to be taken into account. Another example is that some types of chemical gas sensors are power-hungry and machine learning can be used to replace actual measurements with high-quality predictions. Finally, sensor data coming from IoT sensors measuring gases and particle concentrations in the ambient air help to push the limits of today's air quality maps by extending the conventional networks of static high-quality measurement stations with dense IoT measurements. The challenge is to show that it is possible to build high-quality and high-resolution air quality maps using low-cost, less precise, low-resolution, less stable, noisy sensors. While successfully solving this challenge, we managed to improve the accuracy of air quality models by accounting for air pollution transfer, and used our models to understand the impact of COVID-19 lockdown measures on local air quality.

Recent publications (see full list here):

Involved researchers:

Scientific publications
image/svg+xml

For a complete list of the scientific publications of our group, click here or check the PURE information here.

External collaborations
image/svg+xml
Awards (last five years)
image/svg+xml