Sparse Gaussian Process Emulators for Surrogate Design Modelling


Article Preview

Efficient surrogate modelling of computer models (herein defined as simulators) becomes of increasing importance as more complex simulators and non-deterministic methods, such as Monte Carlo simulations, are utilised. This is especially true in large multidimensional design spaces. In order for these technologies to be feasible in an early design stage context, the surrogate model (oremulator) must create an accurate prediction of the simulator in the proposed design space. Gaussian Processes (GPs) are a powerful non-parametric Bayesian approach that can be used as emulators. The probabilistic framework means that predictive distributions are inferred, providing an understanding of the uncertainty introduced by replacing the simulator with an emulator, known as code uncertainty. An issue with GPs is that they have a computational complexity of O(N3) (where N is the number of data points), which can be reduced to O(NM2) by using various sparse approximations, calculated from a subset of inducing points (where M is the number of inducing points). This paper explores the use of sparse Gaussian process emulators as a computationally efficient method for creating surrogate models of structural dynamics simulators. Discussions on the performance of these methods are presented along with comments regarding key applications to the early design stage.



Edited by:

Peter F. Pelz and Peter Groche




P. Gardner et al., "Sparse Gaussian Process Emulators for Surrogate Design Modelling", Applied Mechanics and Materials, Vol. 885, pp. 18-31, 2018

Online since:

November 2018


* - Corresponding Author

[1] Jerome Sacks, William J. Welch, Toby J. Mitchell, and Henry P. Wynn. Design and Analysis of Computer Experiments. Statistical Science, 4(4):409-423, (1989).


[2] Anthony O'Hagan. Bayesian analysis of computer code outputs: A tutorial. Reliability Engineering & System Safety, 91(10-11):1290-1300, oct (2006).


[3] Marc C. Kennedy and Anthony O'Hagan. Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(3):425-464, (2001).


[4] Philipp Hennig and Christian J. Schuler. Entropy Search for Information-Efficient Global Optimization. Journal of Machine Learning Research, 13:1809-1837, dec (2012).

[5] Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian Optimization of Machine Learning Algorithms. In NIPS, pages 2951-2959, jun (2012).

[6] Ioannis Andrianakis, Ian R. Vernon, Nicky McCreesh, Trevelyan J. McKinley, Jeremy E. Oakley, Rebecca N. Nsubuga, Michael Goldstein, and Richard G. White. Bayesian History Matching of Complex Infectious Disease Models Using Emulation: A Tutorial and a Case Study on HIV in Uganda. PLoS Computational Biology, 11(1), (2015).


[7] Paul Gardner, Charles Lord, and Robert J. Barthorpe. Bayesian History Matching for Forward Model-Driven Structural Health Monitoring. In Proceedings of IMAC XXXVI, (2018).


[8] Anthony O'Hagan and John F. C. Kingman. Curve Fitting and Optimal Design for Prediction. Journal of the Royal Statistical Society: Series B (Statistical Methodology) (Statistics in Society), 40(1):1-42, (1978).

[9] Carl E. Rasmussen and Christopher K. I. Williams. Gaussian processes for machine learning., volume 14. (2004).

[10] Joaquin Quiñonero-Candela and Carl Edward Rasmussen. A Unifying View of Sparse Approximate Gaussian Process Regression. Journal of Machine Learning Research, 6:1939-1959, (2005).

[11] Edward Snelson and Zoubin Ghahramani. Local and global sparse Gaussian process approximations. In 11th International Conference on Artificial Intelligence and Statistics, (2007).

[12] Thang D. Bui, Josiah Yan, and Richard E. Turner. A Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation. Journal of Machine Learning Research, 18(1):3649-3720, (2017).

[13] Edward Snelson and Zoubin Ghahramani. Sparse Gaussian Processes using Pseudo-inputs. In NIPS, pages 1257-1264, (2005).

[14] Michalis Titsias. Variational Learning of Inducing Variables in Sparse Gaussian Processes. In AISTATS, volume 5, pages 567-574, (2009).

[15] James Hensman, Nicolo Fusi, and Neil D. Lawrence. Gaussian Processes for Big Data. In Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI2013), pages 282-290, sep (2013).

[16] Hildo Bijl, Jan Willem van Wingerden, Thomas B. Schön, and Michel Verhaegen. Online sparse Gaussian process regression using FITC and PITC approximations. IFAC, 48(28):703-708, (2015).


[17] Noel Cressie. Statistics for Spatial Data. Terra Nova, 4(5):613-617, (1992).


[18] Ioannis Andrianakis and Peter G Challenor. The effect of the nugget on Gaussian process emulators of computer models. Computational Statistics & Data Analysis, 56(12):4215-4228, dec (2012).


[19] Leonardo S. Bastos and Anthony O'Hagan. Diagnostics for Gaussian Process Emulators. Technometrics, 51(4):425-438, (2009).


[20] Matthias Bauer, Mark van der Wilk, and Carl Edward Rasmussen. Understanding Probabilistic Sparse Gaussian Process Approximations. In NIPS, pages 1533-1541, (2016).

[21] Rik Pintelon and Johan Schoukens. System Identification: A Frequency Domain Approach. (2012).