Mobile Sensing & Mapping

The automated recognition and geo-referencing of objects in the vincinity of rail/road infrastructure.

IPOS - Object Detection Beneath the Train

The accurate and reliable detection and localization of objects on rail areas is challenging, due to the large area to be covered, the potentially small object size and the limited measurement time available in between regular train operation. In this project we propose a measurement system capable of continuously monitoring the railway cross-section beneath a train, geo-referencing the data and performing object classification and unusual object detection.


 We rely on high-speed Laser and multi-camera systems to continuously scan the entire railway cross-section beneath a train. Measurement speed is designed for a continuous operation at 25m/s. Data is acquired on the train, and fused to a continuous image and 3D model during post-processing.

 By fusing the captured image data with GNSS, odometry and IMU data we are able to geo-reference the acquired image data itself, as well as object detections automatically performed on the data.

The acquired image data provides metric 3D information at millimeter-resolution. With this data stream, we perform automated detection of specific objects of interest, as well as a continuous search for unexpected objects which may interfere with railway operations.


Contact: Matthias RĂ¼ther

To top

SENS - Detection and Geo-Localization of Rail Infrastructure Elements

Knowledge of the exact geo-location of individual elements within a traffic infrastructure system helps in routine maintenance and inventarization. In this project we developed a multi-camera system which is mounted on a locomotive and able to detect specific objects around the railway system. In combination with GNSS receivers, odometer and IMU system, we are able to geo-localize detected objects with mm accuracy.

The sensor system is scalable and consists of two or four cameras mounted in an overlapping stereo configuration. The weather-proof design and active illumination allow for continuous 24-hour operation. High-dynamic range imaging allows to cope with specular highlights and very dark objects as well. An object resolution of 0.3mm / pixel at a framerate of 70fps allow for precise object recognition and localization, even at a travel speed of 25m/s.
The camera setup is complemented by powerful onboard compute capacity and sensors for precise geo-referncing. Multiple GNSS antennas combined with an aerial inertial measurement unit and odometer system precisely locate the sensor carrier in space.


Masts beside the railway are automatically identified. If requested, objects mounted on the mast like reference markers are also automatically detected, segmented and triangulated for a millimeter-accurate location. Posteriuor data fusion with GNSS information allow us to transfer these local coordinates into a geographic coordinate frame.


Writings and kilometer-markings on the mast are automatically detected and recorded.

To top

Youtube Channel


2016/12/16: New Open Student Position: LIDAR Metrology Tooling

--> Learn More

2016/12/01: New Open Student Position: Robotic Charging of Electric Vehicles

--> Learn More

2016/07/15: Accepted to BMVC 2016

Our paper "A Deep Primal-Dual Network for Guided Depth Super-Resolution" has been accepted for oral presentation at the British Machine Vision Conference 2016 held at the University of York, United Kingdom.

2016/07/11: Accepted to ECCV 2016

Our paper "ATGV-Net: Accurate Depth Superresolution" has been accepted at the European Conference on Computer Vision 2016 in Amsterdam, The Netherlands.

2015/10/07: Accepted to ICCV 2015 Workshop: TASK-CV

Our paper "Anatomical landmark detection in medical applications driven by synthetic data" has been accepted at the IEEE International Conference on Computer Vision 2015 workshop on transferring and adapting source knowledge in computer vision.

2015/09/14: Camera calibration code online

The camera calibration toolbox accompanying our paper "Learning Depth Calibration of Time-of-Flight Cameras" is available here.

2015/09/07: Accepted to ICCV 2015

Our papers "Variational Depth Superresolution using Example-Based Edge Representations" and "Conditioned Regression Models for Non-Blind Single Image Super-Resolution" have been accepted at the IEEE International Conference on Computer Vision 2015, December 13-16, Santiago, Chile.

2015/07/03: Accepted to BMVC 2015

Our papers "Depth Restoration via Joint Training of a Global Regression Model and CNNs" and "Learning Depth Calibration of Time-of-Flight Cameras" have been accepted as a poster presentation at the 26th British Machine Vision Conference, September 7-10, Swansea, United Kingdom.