BIVISION: Bilevel Optimization for Computer Vision


Variational methods are among the most successful methods to solve inverse problems in computer vision and image processing. Typical problems are tasks such as image restoration, motion estimation, stereo and 3D reconstruction. Existing variational models in computer vision are mainly hand-designed based on simple principles derived from the intrinsic properties of images. Clearly, these models are often too simple to model the complex physical properties of the visual world. In this project, we make a significant step ahead by considering more complex variational models based on higher-order, non-local and data-adaptive regularization and propose to learn the involved model parameters using optimization methods. The main idea is to learn the parameters of the variational models, such that the solution of the variational model minimizes a certain loss function that measures the error between the ground truth solutions and the solutions predicted by the model. This problem naturally leads to a bilevel optimization problem, where the lower level problem is given by the variational model and the higher level problem is given by the loss function. It turns out that these bilevel optimization problems have many interesting properties, which are still too less investigated in order to make the method accessible for a larger community. Therefore, it is the main goal in this project to develop a unified framework that can be applied to a number of variational problems in computer vision. The unified learning framework will allow us to systematically investigate existing models as well as new models, leading to a better understanding of their advantages and limitations. We expect that the results of this project will lead to new models that can be optimized towards specific applications and image data and hence will perform significantly better than existing models.

Selected Topics

Variational Networks: Connecting Variational Methods and Deep Learning
Trainable Regularization for Multi-frame Superresolution
Learning a variational network for accelerated MRI reconstruction
Joint Demosaicing and Denoising using multi-level optimization to learn model parameters
From grid search to continuous model selection using bi-level optimization
Project partners