Bridging a Gap in Independent Component Analysis

Abstract:

Independent Component Analysis (ICA) is a powerful method which aims at representing a given random signal as a sum of independent sources. The engineering community, however, works at the distribution function or characteristic function level while makes assertions at the random variable level. This legitimacy of this jump has never been established and consists of a longstanding gap in the ICA literature. In this paper, it is proved that existence of a factorization of characteristic function does imply existence of a corresponding decomposition of random variable into independent sum and thus the gap is bridged. The proof relies on two nontrivial results from probability theory.

• Periodical: Applied Mechanics and Materials (Volumes 239-240)
• Pages: 1279-1283
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.239-240.1279
• Online since: December 2012
• Authors: Guo Hua Dong, Yan Li, De Wen Hu
• Keywords: Decomposition of Random Variable, Factorization of Characteristic Function, GaP, Independent Component Analysis (ICA)

References

[1] A. Hyvärinen, J. Karhunen, and E. Ojá: Independent Component Analysis (John Wiley & Sons, 2001).

[2] A.J. Bell and T.J. Sejnowski: Vision Res. Vol 37 (1997), pp.3327-3338

[3] J.H. van Hateren and A. van der Schaaf: Proc. Royal Soc. London, Ser. B Vol 265 (1998), pp.359-366

[4] B.A. Olshausen and D. Field: Nature Vol 381 (1996), p.607–609

[5] D.J. Field: Phil. Trans. Royal Soc. London, Ser. A, Vol 357 (1999), pp.2527-2542

[6] I. Daubechies, E. Roussos, S. Takerkart, M. Benharrosh, C. Golden, K. D'Ardenne, W. Richter, J.D. Cohen, and J. Haxby: PNAS Vol. 106 (2009), pp.10415-10422

DOI: 10.1073/pnas.0903525106

[7] Y. Amit, D. Geman, and K. Wilder: IEEE Trans. Patt. Anal. Mach. Intell. Vol 19 (1997), p.1300–1305

[8] R. Battiti: IEEE Trans. Neu. Net. Vol 5 (1995), pp.537-550

[9] B. V. Bonnlander and A.S. Weigend, in: Proc. of the 1994 Symp. on Artificial Neu. Net. (ISANN 94) (1996), pp.42-50

[10] K. Torkkola: J. Mach. Learning Res. Vol 3 (2003), p.1415–1438

[11] M. Vidal-Naquet and S. Ullman: in: Proc. Int. Conf. Comp. Vision 2003 (2003), p.281–288

[12] F. Fleuret: J. Mach. Learning Res. Vol 5 (2004), p.1531–1555

[13] M. Bressan and J. Vitrià: IEEE Trans. Patt. Anal. Mach. Intell. Vol. 25 (2003), pp.1312-1316

[14] P. Comon: Sig. Proc. Vol. 36 (1994), pp.287-314 Special issue on higher-order statistics

[15] A. Haddad and Y. Meyer: Variational Methods in Image Processing. Preprint CMLA 2005-08

[16] J.-F. Cardoso's Homepage: http://sig.enst.fr/~cardoso/stuff.html

[17] G.H. Dong and D.W. Hu. On Generic Impossibility of ICA Decomposition. (Submitted to IScIDE 2012)

[18] W. Feller: An Introduction to Probability Theory and Its Applications, vol. 2 (John Wiley & Sons, 2nd ed., 1971)

[19] A.M. Kagan, Yu.V. Linnik, and C. R. Rao: Characterization Problems in Mathematical Statistics (John Wiley & Sons, 1973)

[20] Yu. V. Linnik: Decomposition of Probability Laws (Oliver and Boyd Ltd., Edinburgh, 1964)

[21] E. Lucas: Characteristic Functions (Charles Griffin & Company Ltd., 1960)

[22] G.M. Fel'dman: Arithmetic of Probability Distributions and Characterization Problems on Abelian Groups. Translations of Mathematics Monographs, vol. 116 (Simeon Ivanov, ed.), (American Mathematical Society, Providence, RI, 1993)

DOI: 10.1090/mmono/116/03

[23] K.R. Parthasarathy, R. Ranga Rao, and S.R.S. Varadhan: Trans. AMS Vol. 102 (1962), pp.200-217

[24] M. Kac: Statistical Independence in Probability, Analysis and Number Theory. (Published by the Mathematical Association of America, and Distributed by John Wiley and Sons, Inc., 1959)

[25] O. Kallenberg: Foundations of Modern Probability (Springer-Verlag, 1997)

[26] R.M. Dudley: Ann. Math. Stat. Vol. 39 (1968), pp.1563-1572

[27] R. M. Dudley: Real Analysis and Probability (Cambridge University Press, 2004)

[28] S.T. Rachev: Probability Metrics and the Stability of Stochastic Models (John Wiley & Sons, 1991)

[29] P. J. Huber: Robust Statistics (John Wiley & Sons, New York, 1981)