Residual Treed Gaussian Processes

Abstract: Gaussian processes (GP) enjoy wide popularity in spatial statistics, uncertainty quantification, and machine learning. With the advance of measurement technologies and increasing computing power, large numbers of measurements and large-scale numerical simulations make traditional GP models and computational strategies inadequate in dealing with spatially heterogeneous and big data, especially in multi-dimensional domains. In recent years, several multi-scale or tree-based extensions of the GP have been introduced to model spatial nonstationarity and/or achieve scalable computation. In this talk, I introduce a new Bayesian tree-based GP inference framework called residual treed GP (ResTGP). ResTGP combines key features of the treed GP (Gramacy and Lee 2008) and the multi-resolution GP (Fox and Dunson 2012), thereby enjoying the computational efficiency of the formal and the flexibility of the latter. Our main idea is to decompose a Gaussian process as well as the data at a cascade of resolutions across locations through iteratively computing predictive and residual processes, thereby characterizing the underlying covariance structure and achieving divide-and-conquer on the data points simultaneously. We also introduce a new computational strategy for Bayesian inference for ResTGP that does not rely on Metropolis-Hastings based stochastic tree search algorithms but is based on recursive message passing. This is joint work with Li Ma at Duke University.