Aapo Hyvärinen and Patrik Hoyer
The Computational Neuroscience Group
We have introduced modifications of the basic ICA model that describe further aspects of natural image statistics. The modifications use a linear decomposition as in the basic ICA case, but the components are not assumed to be all independent.
The first model in this direction is independent subspace analysis, in which the components are divided into groups or subspaces so that components in different subspaces are independent, but components in the same subspace are not. In particular, the distribution of the components in a subspace is assumed to depend only on the norm of the projection on that subspace. Typically, this implies that the components of a subspace tend to be active simultaneously.
When estimated from natural image data, the model shows emergence of complex cell properties, in particular phase and translation invariance (insensitivity), together with orientation and frequency selectivity. Here are the estimated basis vectors, grouped according to the subspace structure.
Furthermore, we have generalized the independent subspace model so that it models more general dependency structures.
The point is to define a topographic order using the higher-order correlations of the components. Basically, we use correlations of energies, i.e. squares, of the components. Thus we order the basis vectors so that component that are near-by in the topographic representation tend to be active, i.e. non-zero at the same time. This can be considered as a generalization of independent subspaces, so that every neighbourhood corresponds to one subspace. Thus we obtain a linear representation in which the coefficients and the basis vectors have a topographic organization that gives us information on the statistical higher-order structure of the data.
When estimated from natural image data, the model shows simultaneous emergence of topography (i.e. spatial organization on the cortical surface) and complex cell properties. This is because every neighbourhood corresponds to one feature subspace as in independent subspace analysis, i.e. one complex cell.
For details, see the articles available on the publication pages of
Aapo Hyvärinen
and Patrik Hoyer.
Some data we are currently using can be found here.
We also distribute a MATLAB package for estimating ICA, ISA, and TICA bases from image data!