Noisy ICA and Applications to Image Denoising

Aapo Hyvärinen and Patrik Hoyer


In this project, we have done research on how to estimate the noisy version of the ICA model. This means that the observed mixtures are corrupted by additive gaussian noise. Ordinary ICA methods cannot be used in such a case. One approach is to modify the FastICA algorithm so that it is immune to noise. This was presented in:

A. Hyvärinen. Gaussian Moments for Noisy Independent Component Analysis.
IEEE Signal Processing Letters, 6(6):145--147, 1999.
Abstract  Postscript  gzipped PostScript.

This modification was based on the concept of gaussian moments, which are non-polynomial moments that are in a certain respect immune to noise (or rather, the effect of noise on the estimators can be precisely controlled).

A second approach is to consider the joint likelihood of the mixing matrix and the independent components. See this paper:

A. Hyvärinen. Independent Component Analysis in the Presence of Gaussian Noise by Maximizing Joint Likelihood. Neurocomputing, 22:49-67, 1998.
Abstract  Postscript  gzipped PostScript.

The advantage with this method is that one can derive optimal estimates of the independent components as well. This leads in fact to a denoising method, described next.

Sparse Code Shrinkage for Image Denoising

We have developed a method we call sparse code shrinkage for denoising. It is based on using the ICA representation of the data, e.g. images, which is closely connected to so-called sparse coding. For information on what the ICA representation of images looks like, see this page. Using maximum likelihood estimation of the noise-free independent components, we obtain a method that is quite similar to wavelet shrinkage and coring methods.

Here is an example of the obtained results. Top-left: the original image (standard deviation of pixels is 1.0). Top-right: Gaussian noise of standard deviation 0.5 added. Bottom-left: classic wiener filter restoration. Bottom-right: Sparse Code Shrinkage.

\resizebox {.4\textwidth}{!}{\includegraphics{results/papu.ps}}\resizebox {.4\textwidth}{!}{\includegraphics{results/papuN5.ps}}
\resizebox {.4\textwidth}{!}{\includegraphics{results/papuN5wien.ps}}\resizebox {.4\textwidth}{!}{\includegraphics{results/papuN5scs.ps}}

For more details, see the articles available on the publication pages of
Aapo Hyvärinen and Patrik Hoyer

Aapo Hyvarinen
15 Feb 2000