Skip to content

Emulation for stochastic models

Stochastic models are often computationally extensive, thereby limiting the number of simulations that can be run as part of a calibration exercise considering realistic computational budgets. Therefore, we may seek to learn as much as possible from each simulation run. Emulation is one approach to achieve this goal.

An emulator is a statistical model that approximates the relationship between model inputs and outputs. Emulators are often used to speed up calibration by providing a fast approximation of the model outputs or likelihood function, thereby allowing for more efficient exploration of the parameter space.

These methods work by "learning" from a set of training simulations and then predicting outputs or likelihoods at new parameter settings. Often, emulators borrow information from nearby simulations in parameter space to make predictions, thereby leveraging the smoothness of the underlying function being approximated.

Most emulators have parameters that must be learned, these are called hyperparameters.

There are two main approaches that are commonly used in practice:

  1. Directly emulate the loss (e.g. likelihood or log-likelihood).
  2. Emulate individual model outputs, and then use those to calculate the loss.

Warning

Because emulators borrow information from nearby simulations, they often introduce bias into the calibration process. If used directly in algorithms such as MCMC, this bias can lead to incorrect posterior distributions. The result is that the posterior samples will not be distributed perfectly according to the true loss, a guarantee that comes from unbiased estimation. Instead, what you’ll get is the "posterior" under the approximate loss surface defined by the emulator. In practice, however, a well-trained emulator can still provide such dramatic performance improvements that the bias in posterior samples may be acceptable. If you're worried about bias, avoid emulation or perform some tests to assess if the bias is acceptable.

Bias-avoiding techniques have been developed as well. Once such technique is delayed acceptance MCMC 1 2, which uses the emulator to screen out poor proposals before running the full simulation.

There are many techniques for emulation including artificial neural networks, Bayes linear regression, and Gaussian processes (GPs). GPs are particularly popular due to their flexibility and ability to quantify uncertainty.

But please note that emulators make assumptions about the shape of the function being evaluated. Careful selection of hyperparameters and rigorous out-of-sample validation are essential to ensure that the emulator generalizes well to unseen parameter settings.

References

  1. Andrieu, C., & Roberts, G. O. (2009). The pseudo-marginal approach for efficient Monte Carlo computations. Annals of Statistics, 37(2), 697–725. https://doi.org/10.1214/07-AOS574 ↩

  2. Andrieu, C., Doucet, A., & Holenstein, R. (2010). Particle Markov chain Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3), 269–342. https://doi.org/10.1111/j.1467-9868.2009.00736.x ↩