An Introduction to Approximate Bayesian computation
Wed 10 January 2018
Bayesian methods are increasingly valuable to scientists, given their flexibility, ease of use and intuitive nature, as compared to more classical 'frequentist' methods (they're also much more easily used naively, which is good, as many scientists are not trained statisticians). However, Bayesian methods are often quite computationally expensive, typically due to the computation of something called the posterior (more on this below). In this post, I will be giving a brief overview of how Bayesian methods can be used in the process of model selection, i.e., selecting the best mathematical model for a given process or phenomenon from a group of models. Then, I will be looking at methods that can circumvent the difficulties that arise when you don't have a 'closed-form', analytical probability distribution for your data based on your model (this is called the model likelihood, by the way), since many Bayesian computations depend on you having this information at hand. So, without further ado, let's get started!
A 10-Second Tutorial on Bayesian Model Selection
(More of this can be found at Jake VanderPlas' blog post on the same subject)
- I want to find the model, \(M\), that best describes my data. This is different from finding the best parameterization of my model, which assumes that I already have the best model (though this is another important statistical problem and is called model fitting.
- I can compare my models using something called the model posterior, that is, \(P(M_{i}|D)\). This is the probability of model \(M\) given that we know our data (this is as opposed to using the model likelihood \(P(D|M)\) to evaluate models, as was explained above)
- The model posterior can be obtained from the model likelihood using Bayes' rule, that is, \(P(M|D) = \frac{P(D|M)P(M)}{P(D)}\)
- We actually compare models using the Bayes factor, ie, , since we \(P(D)\) is not often well-defined Many statistical problems do have direct access to a closed-from likelihood function. However, many do not. This is where Approximate Bayesian computation comes in!