Sparse Incremental Delta-Bar-Delta for System Identification

Abstract:

Metalearning algorithm learns the base learning algorithm, targeted for improving the performance of the learning system. The incremental delta-bar-delta (IDBD) algorithm is such a metalearning algorithm. On the other hand, sparse algorithms are gaining popularity due to their good performance and wide applications. In this paper, we propose a sparse IDBD algorithm by taking the sparsity of the systems into account. Thenorm penalty is contained in the cost function of the standard IDBD, which is equivalent to adding a zero attractor in the iterations, thus can speed up convergence if the system of interest is indeed sparse. Simulations demonstrate that the proposed algorithm is superior to the competing algorithms in sparse system identification.

• Periodical: Applied Mechanics and Materials (Volume 665)
• Pages: 643-646
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.665.643
• Online since: October 2014
• Authors: Ying Liu*, Yan Ye, Chun Guang Li
• Keywords: Adaptive Filter, Incremental Delta-Bar-Delta, l₀ Norm, Least Mean Square, Metalearning, Sparse, System Identification

References

[1] R.A. Jacobs, Increased rates of convergence through learning rate adaptation, Neural Networks1(1988) 295-307.

DOI: 10.1016/0893-6080(88)90003-2

[2] R.S. Sutton, Adaptive bias by gradient descent: an incremental version of Delta-Bar-Delta, In: Proc. the Tenth National Conference on Artificial Intelligence (1992) 171-176.

[3] W.F. Schreiber, Advanced television systems for terrestrial broadcasting: some problems and some proposed solutions, Proc. IEEE83 (1995) 958-981.

DOI: 10.1109/5.387095

[4] D.M. Etter, Identification of sparse impulse response system using an adaptive delay filter, In: Proc. ICASSP (1985).

[5] M. Godavarti, A.O. Hero, Partial update LMS algorithms, IEEE Trans. Signal Process. 53 (2005) 2382-2399.

DOI: 10.1109/tsp.2005.849167

[6] R. Tibshirani, Regression shrinkage and selection via the LASSO, J. Royal Statistical Society, 58 (1996) 267-288.

DOI: 10.1111/j.2517-6161.1996.tb02080.x

[7] Y. Gu, J. Jin, and S. Mei, l0 norm constraint LMS algorithm for sparse system identification, IEEE Signal Process. Lett. 16 (2009) 774-777.

DOI: 10.1109/lsp.2009.2024736

[8] Y. Chen, Y. Gu,A.O. Hero, Sparse LMS for system identification, In: Proc. ICASSP (2009).

[9] Y. Liu, C.G. Li,Z.Y. Zhang, Diffusion sparse least-mean squares over networks, IEEE Trans. Signal Process. 60 (2012) 4480-4485.

DOI: 10.1109/tsp.2012.2198468

[10] P.S. Bradley, O.L. Mangasarian, Feature selection via concave minimization and support vector machines, In: Proc. 13th ICML (1998).