9. Estimation

We’ve seen some examples now of how to get the posterior distributions of our models. Now we’re asking: what is the meaning of the posterior and what can we do with it? We need to be able to summarize it in useful ways and make decisions based on these summaries.

Point estimates are the simplest summary of the posterior. We’ve already used the posterior mean in previous examples, but let’s back up a bit first and think about a more general framework.

Bayesian decision theory

The professor touches on the concepts of actions and risk from Bayesian decision theory. An action is a specific estimation or decision made based on the posterior distribution. These decisions can be evaluated by considering the loss or error associated with that decision. The Bayes action will be a point estimate that minimizes the chosen loss function.

Note

You’ll often see Bayesian literature refer to the “utility function,” rather than the loss function. The utility function is the negated loss function; it measures something positive that we want to maximize rather than minimize. For more information on this and other topics in this lecture, see Robert [2007] Chapter 2: Decision-Theoretic Foundations. The author also has hundreds of posts on Stack Exchange under the username Xi’an.

The lecture goes over some of the most common loss functions and their connections to the posterior median and mode. If you want to see the full derivations for these, this Statistics Matt video has them. But in practice, we already know the equations that give means, medians, and modes for many distributions, unless they don’t have a closed-form solution. Once we start using MCMC methods, our posteriors will be represented by arrays of samples, so finding an estimate of the mean can be as simple as np.mean(samples).