Machines learn how to see

These are key questions for brain researchers as well as computer scientists, such as Thomas Pock, who is always looking for international cooperation to achieve progress in image processing. Since 2014, Thomas Pock has held an AIT-endowed professorship for Mobile Computer Vision at the Institute for Computer Graphics and Vision (ICG). His research work, developed in co-operation with colleagues from New York and Paris, focuses on mathematical models to distinguish between “good” images and “bad” images. Ultimately the objective is to filter machine-supplied image signals, extracting only the visual information that is absolutely essential for the reconstruction of a meaningful image with maximum detail.

Research cooperation with New York

One of Thomas Pock’s current projects is to build mathematical models for the reconstruction of two-dimensional images from magnetic resonance imaging signals (MRI signals). His aim is to get the best possible result from the smallest possible amount of signal data. If you need less data, you automatically shorten the scanning time in the MRI, which in turn allows you to increase the number of patients who can be scanned with the machine in one single day and thus reduces costs. In a research co-operation with Florian Knoll and Daniel K. Sodickson from the Department of Radiology at New York University School of Medicine, Thomas Pock and his PhD student Kerstin Hammernik developed an algorithm that does precisely that: it builds highquality images from the undersampled MRI signal data, taking just one sixth of the scan time of previous MRI scans. The research partners in the USA provide the MRI data that are needed to develop this method as well as the know-how in physics and the operating principles of the MRI devices, while Thomas Pock and Kerstin Hammernik devised the mathematical model that reconstructs the images from the undersampled MRI signal data in Graz. One fundamental problem of machine-based image recognition is the sheer endless number of possible images. “If you calculate all theoretical image variations for an image size of no more than 65 times 65 pixels and 256 grey scales, you obtain potentially many more different images than there are atoms in the universe, namely 25^665x65˜10^10000 versus 10^80.” This unimaginably large number of possible images explains simple comparison-based databases cannot work,” explains Thomas Pock.

Machine learning combined with image processing

“As human beings we know what high-quality images look like,” says Thomas Pock. “Now we want computers to do the same – to be able to recognise and classify images within fractions of a second.” In the past, we used to look for mathematical models capable of coping with this massive computing task virtually by hand. Now Thomas Pock combines new methods of machine learning with image processing methods. He designs image models with a large number of degrees of freedom that are able to reconstruct two-dimensional images from the signal data, constantly comparing the result with the ideal image. Thomas Pock adds: “The learning problem involves a loss function that measures the loss and calculates how much the momentarily reconstructed solution differs from the target image, i.e. the appearance of the tissue as reflected in the MRI data. This error is in turn back propagated into the model. This is done by calculating the gradient of the loss function which points in the direction of the strongest change. In this manner the model parameters can be varied to make the mistake smaller.Then you let the process continue until no detectable improvement can be achieved any more.”