A. Hyvärinen. New Approximations of Differential Entropy for Independent Component Analysis and Projection Pursuit.  In Advances in Neural Information Processing System 10 (NIPS*97), pp. 273-279, MIT Press, 1998.
Postscript  gzipped PostScript. The affiliated tech. rep A47: Postscript  gzipped PostScript.

Abstract: We derive a first-order approximation of the density of maximum entropy for a continuous 1-D random variable, given a number of simple constraints. This results in a density expansion which is somewhat similar to the classical polynomial density expansions by Gram-Charlier and Edgeworth. Using this approximation of density, an approximation of 1-D differential entropy is derived. The approximation of entropy is both more exact and more robust against outliers than the classical approximation based on the polynomial density expansions, without being computationally more expensive. The approximation has applications, for example, in independent component analysis and projection pursuit.

Back to my on-line publications