18. Parametric

Lecture errata

Sample variance

The sample variance for the Jeremy’s IQ example here was calculated incorrectly. The variance of the sample \((98, 107, 89, 88, 108)\) is \(S^2 = \frac{(0+9^2+9^2+10^2+10^2)}{(5-1)} = 90.5\). The calculation may be updated as follows:

\[ \hat{B} = \frac{\sigma^2}{\sigma^2 + \hat{\tau}^2} = \frac{80}{80 + (90.5 - 80)} = \frac{80}{90.5} \]

Then the estimated parameters are:

\[ \hat{\theta}_1 = \frac{80}{90.5} \cdot 98 + \frac{10.5}{90.5} \cdot 98 = 98 \]

\[ \hat{\theta}_2 = \frac{80}{90.5} \cdot 98 + \frac{10.5}{90.5} \cdot 107 = 99.044, \text{ etc.} \]

Incorrect conditional on slide 3

The last line of slide 3 in the lecture is:

\[\prod_{i=1}^{n} m_i(x_i|\theta_i)\]

but it should be:

\[\prod_{i=1}^{n} m_i(x_i|\xi)\]

Lecture summary

Empirical Bayes can be categorized into parametric and non-parametric approaches, a taxonomy proposed by Carl Morris in his 1983 JASA paper (Morris [1983]). This lecture primarily focuses on the parametric approach, where the likelihood function and the parameter \(\theta_i\) vary across observations. The individual observations are independent but not identically distributed because of the variation in \(\theta_i\) and the likelihood functions \(f_i\). The priors on \(\theta_i\) are considered to be IID, however.

If the individual observations \(x_i\) are all dependent on a common hyperparameter and share the same likelihood function \(f_i\), then they are marginally IID. Then the hyperparameter, although unknown, can be estimated using the data through a Maximum Likelihood Estimation (MLE) approach.

Empirical Bayes estimates borrow strength from all data points, using the mean and variance of all observations, not just the observation for each test. Hence, instead of analyzing each test separately, the empirical Bayes estimate takes into account all the data points.