Assume two independent sources have been observed in two signals,
pictured below. The left figure pictures their time course. The right
figure is a scatter-plot where, at each time point, the data is
represented as a point in x_{1}-x_{2}-coordinates. The
green line shows the mixing coefficients of the first signal and the
black line of the second signal. Red line shows the projection
direction where the standard deviation of the data is
maximised. Standard deviation of different projection angles have been
depicted by the red nut-shaped curve.

Observed signals |
Scatter plot |

**The aim is to recover the original signals ** when the true
mixing is unknown.

The scatter plot makes it clearly visible that the observed
signals are correlated. An attempt to recover the original sources may
be to seek projections that would render the signals
uncorrelated. This can be done by principal component analysis
(PCA). The first principal component corresponds to the most variating
direction in the data (depicted in red line). In case of 2-dimensional
data, the second component is the minor component and corresponds to
the least variating direction. If the data is projected in the
principal and minor components and their variances are normalised to
unity, the resulting components are uncorrelated and have unit
variance. Those components are called **sphered** (or
whitened). They are pictured below.

Sphered signals |
Scatter plot |

Now the components y_{1} and y_{2} are
uncorrelated. Furthermore, any direction has unit variance. This is
illustrated in the fact that the standard deviation in any direction
is the same. But the directions of the original sources (green and
black lines) do not yet align with the sphered components.

**What to do next?** How to find the correct rotation to
identify the sources?

Sphering has the property that all orthogonal projections result in uncorrelated components. A common name for all uncorrelated projections is factor analysis (FA). If the original sources have Gaussian distributions and the samples are independent and identically distributed (i.i.d.), FA has a rotational ambiguity and nothing more can be done. If not, any of these non-regularities may be used for separating the sources.

Now, let's see. **Do the sphered components have Gaussian
distributions?**

Looks quite Gaussian. No luck there, too bad. **But do the samples
look i.i.d.?** The mean and the variance seem to be quite stable. But
the nearby samples seem to be correlated. To check this, let's look at
a scatter plot where the coordinate pair is made from the consecutive
samples.

**The consecutive samples are clearly correlated.** Could there
be a source that has significant positive correlation between nearby
samples? That would mean a slowly-moving source. How to have a better
estimate of it? How about denoising the whitened components? For
denoising, let's just sum together the consecutive values. How does
the data look then?

**Voilá**, the principal direction of the denoised data aligns
with the first signal direction and we can identify it there. The
second source may be identified with the projection orthogonal to the
first one. In the figure below, the estimated signals are pictured on
the left and their frequency content on the right.

Sphered signals |
Scatter plot |

The first signal seems to be a slowly-moving source, while the second an i.i.d. Gaussian.

You are at: CIS → Linear DSS tutorial

Page maintained by jaakkos@cis.hut.fi, last updated Monday, 21-Mar-2005 11:48:21 EET