Publication Details

Category Text Publication
Reference Category Journals
DOI 10.1002/2014WR016062
Title (Primary) Model selection on solid ground: Rigorous comparison of nine ways to evaluate Bayesian model evidence
Author Schöniger, A.; Wöhling, T.; Samaniego, L. ORCID logo ; Nowak, W.
Source Titel Water Resources Research
Year 2014
Department CHS
Volume 50
Issue 12
Page From 9484
Page To 9513
Language englisch
Keywords Bayesian model selection; Bayesian model evidence; information criteria Abstract
UFZ wide themes RU5;
Abstract

Bayesian model selection or averaging objectively ranks a number of plausible, competing conceptual models based on Bayes' theorem. It implicitly performs an optimal tradeoff between performance in fitting available data and minimum model complexity. The procedure requires determining Bayesian model evidence (BME), which is the likelihood of the observed data integrated over each model's parameter space. The computation of this integral is highly challenging because it is as high-dimensional as the number of model parameters. Three classes of techniques to compute BME are available, each with its own challenges and limitations: 1) Exact and fast analytical solutions are limited by strong assumptions. 2) Numerical evaluation quickly becomes unfeasible for expensive models. 3) Approximations known as information criteria (ICs) such as the AIC, BIC, or KIC (Akaike, Bayesian or Kashyap information criterion, respectively) yield contradicting results with regard to model ranking. Our study features a theory-based intercomparison of these techniques. We further assess their accuracy in a simplistic synthetic example where for some scenarios an exact analytical solution exists. In more challenging scenarios, we use a brute-force Monte Carlo integration method as reference. We continue this analysis with a real-world application of hydrological model selection. This is a first-time benchmarking of the various methods for BME evaluation against true solutions. Results show that BME values from ICs are often heavily biased and that the choice of approximation method substantially influences the accuracy of model ranking. For reliable model selection, bias-free numerical methods should be preferred over ICs whenever computationally feasible.

Persistent UFZ Identifier https://www.ufz.de/index.php?en=20939&ufzPublicationIdentifier=15455
Schöniger, A., Wöhling, T., Samaniego, L., Nowak, W. (2014):
Model selection on solid ground: Rigorous comparison of nine ways to evaluate Bayesian model evidence
Water Resour. Res. 50 (12), 9484 - 9513 10.1002/2014WR016062