General 3D Room Layout from a Single View by Render-and-Compare
We present a novel method to reconstruct the 3D layout of a room—walls, floors, ceilings—from a single perspective view, even for the case of general configurations. This input view can consist of a color image only, but considering a depth map will result in a more accurate reconstruction. Our approach is based on solving a constrained discrete optimization problem, which selects the polygons which are part of the layout from a large set of potential polygons. In order to deal with occlusions between components of the layout, which is a problem ignored by previous works, we introduce an analysis-by-synthesis method to iteratively refine the 3D layout estimate.
This work was supported by the Christian Doppler Laboratory for Semantic 3D Computer Vision, funded in part by Qualcomm Inc.
Given a real image of a room, we first extract depth, semantic segmantion and planar segments in the image. We use this information to infer 3D parameters of planes belonging to the layout components in the image. By intersecting the planes we obtain a list of candidate polygons for each of the layout components. We proceed to find the optimal set of polygons for the layout by solving a discrete optimization problem that reasons both about 2D and 3D quality of the estimated layout. In case of occlusions between layout components, we introduce a Render-and-Compare approach that detects inconsistencies and iteratively refines the layout.