A Neural Network Based on Canonical Correlation for Multicollinearity Diagnosis

Abstract:

We review a recent neural implementation of Canonical Correlation Analysis and show, using ideas suggested by Ridge Regression, how to make the algorithm robust. The network is shown to operate on data sets which exhibit multicollinearity. We develop a second model which not only performs as well on multicollinear data but also on general data sets. This model allows us to vary a single parameter so that the network is capable of performing Partial Least Squares regression to Canonical Correlation Analysis and every intermediate operation between the two. On multicollinear data, the parameter setting is shown to be important but on more general data no particular parameter setting is required. Finally, we develop a second penalty term which acts on such data as a smoother in that the resulting weight vectors are much smoother and more interpretable than the weights without the robustification term. We illustrate our algorithms on both artificial and real data.

• Periodical: Advanced Materials Research (Volumes 756-759)
• Pages: 3324-3329
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.756-759.3324
• Online since: September 2013
• Authors: Ji Fu Nong
• Keywords: Canonical Correlation Analysis, Multicollinearity, Partial Least Squares Regression, Roughness Penalty

References

[1] Diamantaras, K. I., & Kung, S. Y. Multilayer neural networks for reduced-rank approximation. IEEE Transactions on Neural Networks, 1994, 5(5), 684–697.

DOI: 10.1109/72.317721

[2] Fyfe, C. Introducing asymmetry into interneuron learning. Neural Computation, 1995, 7(6), 1167–1181.

[3] Geladi, P., & Kowalski, B. Partial least squares regression: a tutorial. Analytica Chimica Acta, 1986, 185, 1–17.

DOI: 10.1016/0003-2670(86)80028-9

[4] Gou, Z., & Fyfe, C. A family of networks which perform canonical correlation analysis. International Journal of Knowledge-based Intelligent Engineering, 2001, 5(2), 76–82.

[5] Horel, A. E., & Kennard, R. W. Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 1970, 12, 56–67.

DOI: 10.1080/00401706.1970.10488634

[6] Hoskuldsson, A. PLS regression methods. Journal of Chemometrics, 1988, 2, 211–228.

[7] Lai, P. L., & Fyfe, C. A neural network implementation of canonical correlation analysis. Neural Networks, 1999, 12(10), 1391–1397.

DOI: 10.1016/s0893-6080(99)00075-1

[8] Leurgans, S. E., Moyeed, R. A., & Silverman, B. W. Canonical correlation analysis when the data are curves. Journal of the Royal Statistical Society, Series B, 1993, 55(2), 725–740.

DOI: 10.1111/j.2517-6161.1993.tb01936.x

[9] Olshen, R. A., Biden, E. N., Wyatt, M. P., & Sutherland, D. H. Gait analysis and the bootstrap. The Annals of Statistics, 1989, 17, 1419–1440.

DOI: 10.1214/aos/1176347372