ENSEMBLE LEARNING AND LINEAR REPSONSE THEORY FOR ICA
Pedro A.d.F.R. H\ojen-S\orensen, Ole Winther and Lars Kai Hansen
We propose a general framework for performing independent component
analysis (ICA) which relies on ensemble learning and linear response
theory known from statistical physics. We apply it to both discrete and
continuous sources. For the continuous source the underdetermined
(overcomplete) case is studied. The naive mean-field approach fails in
this case whereas linear response theory--which gives an improved
estimate of covariances--is
very efficient. The examples given are for sources without temporal
correlations. However, this derivation can easily be extended to treat
temporal correlations. Finally, the framework offers a simple way of
generating new ICA algorithms without needing to define the prior
distribution of the sources explicitly.
LUTP 00-21