Ensuring driver’s situational awareness is paramount for road safety, particularly when it involves the additional complexities of test driving. Test drivers are tasked with identifying scenarios, classifying them, executing tests, and assessing vehicle responses.
To support these critical tasks effectively, there is a pressing need to develop a fast and responsive test driver support system. This system must seamlessly integrate perception data with the driver’s immediate attention to evaluate relevant signals and provide timely feedback. Moreover, the nuanced attention of test drivers plays a crucial role in data filtering, offering an innovative approach to assigning weak labels to the vast, rapidly accumulated volumes of unlabeled test drive data. In this project, we aim to leverage the potential of 3D glasses to record, detect, and assess the test driver’s attention.
This rich data source will be crucial when ranking the importance of data generated by an in-car automatic data acquisition system, equipped with dual cameras, a microphone, IMU sensors, and other modalities. Your mission is to extract meaningful insights from the test driver’s attention patterns and leverage this knowledge to train a model capable of weakly labeling new data streams. This approach promises to significantly enhance the efficiency and accuracy of our test driving assessments.
Download as PDF
Target Group:
Students in ICE, Computer Science or Software Engineering.
Thesis Type:
Master Project / Master Thesis.
Goals and Tasks:
The project includes the following tasks:
- Literature research on the user gaze detection and attention extraction, weak labeling of multi-modal data streams;
- Data recording with 3D glasses and extraction of attention signals; we will support you with the field data collection using a real test driver setup;
- Aligning attention signals with multi-modal data streams collected with a comprehensive in-car data acquisition system;
- Training a model to predict attention on unseen data streams. Build a video-based demo of the achieved results;
- If time permits, generate attention-aligned textual descriptions of test driving scenarios.
Requirements:
- Eager to work with embedded systems, collect real data and extract valuable insights;
- Programming skills in Python and C/C++;
- Knowledge and interest in applied machine learning; experience with machine learning frameworks (Pytorch, TensorFlow).
Used tools & Equipment:
- A laptop. GPU infrastructure, 3D glasses and data collection support will be provided.
Contact Persons: