V-MAV: Cooperative micro aerial vehicles using onboard visual sensors

The overall goal of the FWF funded VMAV project is to advance the capabilities of small scale flying robots (UAV's) in the areas of flight behavior and flight autonomy. New algorithms will be investigated to make UAV's smarter, more agile and ultimately more useful for applications.

An important focus of the project will be to develop embedded image processing algorithms for real-time localization of UAV's that would enable the UAV to perform dynamic maneuvers. Another focus will be the extraction of semantic information from images and how to use this information for flight planning. In addition cooperative operation of multiple UAV's will be investigated.

To top


Partners

 

The VMAV project is a collaborative DACH project with partners from ETH Zürich, TU München and TU Graz.

Institute for Computer Graphics and Vision, Graz University of Technology 
Institute of Visual Computing, ETH Zürich
Remote Sensing Technology, Technische Universität München

 

 

To top


Project related publications

 

Regularized 3D Modeling from Noisy Building Reconstructions.  Holzmann, Thomas; Fraundorfer, Friedrich; Bischof, Horst. International Conference on 3D Vision. 2016.

Direct Stereo Visual Odometry Based on Lines.  Holzmann, Thomas; Fraundorfer, Friedrich; Bischof, Horst. 11th International Conference on Computer Vision Theory and Applications (VISAPP), 2016. 2016.

A New Paradigm for Matching UAV- and Aerial Images.  Koch, Tobias; Zhuo, Xiangyu; Reinartz, Peter; Fraundorfer, Friedrich. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences. 2016. S. 83-90.

Automatic alignment of indoor and outdoor building models using 3d line segments. Koch, Tobias; Fraundorfer, Friedrich. Beitrag in IEEE/CVF CVPR Workshop on Visual Analysis of Satellite to Street Imagery, USA / Vereinigte Staaten. 2016.

The tum-dlr multimodal earth observation evaluation benchmark. / Koch, Tobias; d'Angelo, Pablo; Kurz, Franz; Fraundorfer, Friedrich; Reinartz, Peter ; Körner, Marco. Beitrag in IEEE/CVF CVPR Workshop on Visual Analysis of Satellite to Street Imagery, USA / Vereinigte Staaten. 2016.

 

 

To top


Activities

Visual Odometry

Our work about "Direct Stereo Visual Odometry based on Lines" won the Best Paper Award at the International Conference on Computer Vision Theory and Applicaitions (VISAPP) in February 2016 in Rome! More details about our work can be found here.

droneSpace inauguration

Our droneSpace at TUG has been finished. It is equipped with an OptiTrack tracking system to control our Pixhawk UAV's.

See below a video of one of the first flight tests.

To top


Software:

Mavmap:

Mavmap is a structure-from-motion system developed by the VMAV partners. It is designed to compute 3D reconstruction from typical UAV imagery.

Mavmap is open-source and hosted on Github (Mavmap repository).

See below an example 3D point cloud computed from Mavmap.

To top


Detailed Project Description

The overall aim of the project is to advance the capabilities of visual controlled MAVs in the areas of flight behavior and autonomy, cooperative operation, cognitive abilities and in addition to decrease the size of such an MAV. Advances in these areas would enable new fields of applications for MAVs and path the way to further research topics in mobile robotics. The proposed research proposal is structured into three work packages: 1. Visual-inertial MAV pose estimation and localization using multi-camera systems 2. Embedded vision algorithms for dynamic flight of small scale MAVs 3. Methods for cooperative visual localization and semantic mapping Work package 1 will investigate the suitability of multi-camera systems for 6DOF pose estimation and localization for MAVs performing dynamic maneuvers. This will include the development of visual-inertial pose estimation algorithms exploiting the advantages of multi-camera system geometries. Work package 2 will investigate embedded computer vision algorithms to facilitate dynamic control and flight as well as a further miniaturization of MAVs. For this, specific components of the visual control system will be moved to dedicated embedded processors to achieve the necessary high-frame rates for dynamic flight. Work package 3 will investigate cooperative operation of MAVs focusing on cooperative visual localization, mapping and cognitive scene understanding and interpretation. In cooperative operation MAVs should be able to share their individual knowledge of the environment and incorporate knowledge of others with the effect of improving environment mapping and the self localization process. An important part of this research package is cognitive scene understanding. The MAVs should make use of object detection and classification methods to generate a semantic description of the environment to produce a semantically annotated 3D environment map and also to use this meta-information to improve the mapping process (e.g. adapt parameters based on the semantics) or the localization process. The proposed project will combine the competences of the three involved partners, ETHZ, TUM and TUG. All the three partners have year-long experience in vision controlled MAV through various projects and performed ground breaking work in this area. The common project will ensure the utilization of the combined expertise of the partners.
Staff member
Participant / Staff Member
Dipl.-Ing. BSc Thomas Holzmann
Funding sources
  • Fonds zur Förderung der wissenschaftlichen Forschung, FWF
External Partners
  • Eidgenössische Technische Hochschule Zürich, ETH
  • Technische Universität München
Start: 01.08.2014
End: 31.01.2018

To top


Project team:
image/svg+xml
Ass.Prof. Dipl.-Ing. Dr.techn. Friedrich Fraundorfer
Dipl.-Ing. BSc Thomas Holzmann