Application of Robust Extracting Outliers Algorithm in Fermentation Process Data Preprocessing

Abstract:

When the process monitoring model based on PCA is established, outliers would destroy covariance structure of model, so monitoring model deviations will affect the monitoring results. Because the CDCm outlier detection algorithm based on robust scaling underestimates the mean and standard deviation of the normal process data, some outliers will be not detected. The CDCm outlier detection algorithm based on modified scaling achieves an accurate estimate for the original data and can overcome the lack of robust scaling by using the maximum variable value to calculate the distance, using CDCm algorithm to obtain the shortest normal point between sampling values and center distance, which will obtain valid data, computing the first Markov distance of iterative algorithm, and by selecting the samplings corresponding the smaller distance value for iteration to obtain normal data. The robust scaling and modified scaling detection algorithm is applied to fermentation process and the test results have shown that modified scaling CDCm detection algorithm is more accurate and effective.

• Periodical: Applied Mechanics and Materials (Volumes 130-134)
• Pages: 463-466
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.130-134.463
• Online since: October 2011
• Authors: Xin Rong Zhang, Bo Chang
• Keywords: Data Preprocessing, Fermentation Process, Outlier, Robust

References

[1] Hawkins D. Identification of Outlier[M]. London: Chapman and Hall Press, (1980).

[2] WANG Hong-ding, TONG Yun-hai, TAN Shao-hua, etc. Research progress on outlier mining[J]. Caai Transactions on Intelligent Systems, 2006, 1(1): 67-73.

[3] PEI Li-zhi, WANG Run-sheng. Moving shadow detection method based on PCA[J]. Journal of Computer Applications, 2009, 29(10) : 2687-2689.

DOI: 10.3724/sp.j.1087.2009.02687

[4] Jolliffe I T. Principal Component Analysis[M]. New York: Springer Verlag Press, 1986. 23-44.

[5] Pearson P K. Exploring process data[J]. Journal of Process Control, 2001(11): 179-194.

[6] Rousseeuw P J, Croux C. Alternatives to the median absolute deviation[J]. Am Stat Assoc, 1993(88): 1273-1283.

DOI: 10.1080/01621459.1993.10476408

[7] William J E, Stephen L M. Outlier detection in multivariate analytical chemical data[J]. Analytical Chemistry, 1998, 70(11): 2372-2379.

DOI: 10.1021/ac970763d

[8] Helge H, Liang Y , Kvalheim O M. Trimmed object projections: a nonparametric robust latent-structure decomposition method[J]. Chemometrics and Intelligent Laboratory Systems, 1995(27): 33-40.

DOI: 10.1016/0169-7439(95)80005-t

[9] Croux C, Ruiz-Gazen A. in: COMPSTAT[C]. Proceedings in Computational Statistics, 1996. 211-217.

[10] Leo H C, Randy J P, Mary B S. Exploring process data with the use of robust outlier detection algorithms[J]. Journal of Process Control, 2003(13): 437-449.

DOI: 10.1016/s0959-1524(02)00068-9