2. Reasons to Use Hierarchical Models#
Why the heirarchy?#
Modeling requirement. For example, a meta-analysis
Prior information can be separated into structural and subjective/noninformative part (like this example)
Robustness + objectivity, let the data “talk” about the hyperparameters
Computing issues (MCMC efficiency, perhaps to a lesser extent now than in the past)
Example#
Posterior with single parameter \(\theta\):
Posterior with an independent \(\theta_i\) for each \(y_i\):
Now let’s create a heirarchical posterior with a \(\theta_i\) for each \(y_i\), and shared prior \(\phi\) on the \(\theta_i\)’s. The \(\theta_i\)’s are no longer independent, but exchangeable, meaning the order of the \(y_i\)’s does not impact the posterior distribution.
If \(y_i\)’s are independent, then they are automatically exchangeable, because of the communative property when they are multiplied together in the joint distribution.
However, exchangeable variables may be dependent. For example, the 2-dimensional multivariate Normal,
the 2 variable \(X\) and \(Y\) are not independent, they are correlated, and yet the are exchangeable because the distribution of \((Y,X)\) is the same as the distribution of \((X,Y)\).