In this project, we proposed methods for urban 3D reconstruction, which generate visually appealing (i.e., building models having planar surfaces and sharp edges, surroundings reconstructed as smooth surface) and compact reconstructions from urban environments using images acquired by Micro Aerial Vehicles (MAVs) as input.
We investigated in 3 methods, where the first one is a slicing-based approach which delivers simplified building models of buildings following a specific geometry [3,4]. In the next method , we tried to overcome the restrictions on specific geometric shapes of buildings and incorporated a plane prior within a generic 3D reconstruction framework. Planes are detected in the point cloud and used within the final 3D optimization framework. With this approach, it is possible to reconstruct any geometric structure, while planar surfaces get reconstructued completely planar, which makes it easier to simplify the model and makes it more visually appealing (less reconstruction noise). Our most recent approach  additionally incorporates semantic information and a 3D line reconstruction within the overall reconstruction process. Using semantics, different scene parts can be treated differently (e.g., building parts should contain a plane prior, vegetation parts should not contain a plane prior). Further, we proposed a plane detection algorithm using 3D line segments as input, which significantly improved the plane detection results on poorly textured areas like facades.
See below a video of results from our most recent approach. Results from our other approaches can be found in the papers and the corresponding videos.
This research was funded by the Austrian Science Fund (FWF) in the project V-MAV (I-1537), and by the Austrian Research Promotion Agency (FFG) in the project FreeLine (Bridge1/843450) and the project Construct (830035).
 Semantically Aware Urban 3D Reconstruction with Plane-Based Regularization Holzmann, T., Maurer, M., Fraundorfer, F., Bischof, H. 2018 in: European Conference on Computer Vision (ECCV) [PDF][Supp. Material][Video]
 Plane-based Surface Regularization for Urban 3D Reconstruction Holzmann, T., Oswald, M.R., Pollefeys, M., Fraundorfer, F., Bischof, H. 2017 in: 28th British Machine Vision Conference (BMVC) [PDF][Supp. Material][Video]
 Regularized 3D Modeling from Noisy Building Reconstructions Holzmann, T., Fraundorfer, F., Bischof, H. 2016 in: 4th International Conference on 3D Vision (3DV) [PDF][Video]
 Geometric Abstraction from Noisy Image-Based 3D Reconstructions Holzmann, T., Hoppe, C., Kluckner, S., Bischof, H. 2014 in: Workshop of the Austrian Association for Pattern Recognition (ÖAGM/AAPR), Best Paper Prize [PDF]