Reliable image reconstruction techniques : enforcing measurement consistency in deep learning methods
Abstract
Solving linear inverse problems (LIPs) is an integral process within many scientific
fields. These problems typically involve attempting to reconstruct a model or signal
from a set of observations or effects. In practise, LIPs can take many different forms.
These encompass anything from a magnetic resonance imaging (MRI) scan, where
a detailed internal body image is constructed from radio and magnetic waves, to
calculating Earth’s density from measurements of the gravitational field.
Traditional methods to solve LIPs typically involve optimization algorithms, which
have been rigorously studied for decades. However, modern deep learning (DL)
networks have now almost completely taken over, producing results which are far
superior to previous algorithms. Despite this, deep networks have one critical drawback : they cannot ensure measurement consistency in their outputs. This means
that information present in observed measurements is lost after being fed into a deep
network. The aforementioned optimization algorithms do not suffer from this issue,
which is why they continue to see limited use in applications where measurement
consistency is essential (e.g., reconstruction of medical images).
The central theme of this thesis is to design algorithms that exploit the advantages
of both optimization and DL networks for image reconstruction tasks. First, we
proposed a framework that post-processes the output from a deep network via an
optimization problem. As the proposed method includes an optimization algorithm,
it ensures measurement consistency. Simultaneously, it exploits the exceptional quality of DL networks by encoding their output in the optimization problem. We also
developed an algorithm for solving the proposed optimization problem and showed
how measurement consistency is closely connected with generalization errors. Our
framework was applied to three applications: natural image resolution enhancement,
fusion-based hyperspectral image resolution improvement and MRI reconstruction.
Experiments show that our algorithm achieves the new state-of-the-art in all of the
applications considered, both in image quality and measurement consistency. Our
hybrid framework could be the key to the introduction of the superior performance
of DL networks in applications where measurement consistency is paramount.