Reliable image reconstruction techniques : enforcing measurement consistency in deep learning methods

Abstract

Solving linear inverse problems (LIPs) is an integral process within many scientific fields. These problems typically involve attempting to reconstruct a model or signal from a set of observations or effects. In practise, LIPs can take many different forms. These encompass anything from a magnetic resonance imaging (MRI) scan, where a detailed internal body image is constructed from radio and magnetic waves, to calculating Earth’s density from measurements of the gravitational field. Traditional methods to solve LIPs typically involve optimization algorithms, which have been rigorously studied for decades. However, modern deep learning (DL) networks have now almost completely taken over, producing results which are far superior to previous algorithms. Despite this, deep networks have one critical drawback : they cannot ensure measurement consistency in their outputs. This means that information present in observed measurements is lost after being fed into a deep network. The aforementioned optimization algorithms do not suffer from this issue, which is why they continue to see limited use in applications where measurement consistency is essential (e.g., reconstruction of medical images). The central theme of this thesis is to design algorithms that exploit the advantages of both optimization and DL networks for image reconstruction tasks. First, we proposed a framework that post-processes the output from a deep network via an optimization problem. As the proposed method includes an optimization algorithm, it ensures measurement consistency. Simultaneously, it exploits the exceptional quality of DL networks by encoding their output in the optimization problem. We also developed an algorithm for solving the proposed optimization problem and showed how measurement consistency is closely connected with generalization errors. Our framework was applied to three applications: natural image resolution enhancement, fusion-based hyperspectral image resolution improvement and MRI reconstruction. Experiments show that our algorithm achieves the new state-of-the-art in all of the applications considered, both in image quality and measurement consistency. Our hybrid framework could be the key to the introduction of the superior performance of DL networks in applications where measurement consistency is paramount.