19. Non-parametric

In parametric empirical Bayesian methods, as covered in the prior lecture, data is used to estimate the hyperparameters in the prior. This approach to inference assumes that the parameter of interest comes from a known family of distributions. In non-parametric empirical Bayes, however, there is no assumption about the family that the parameters come from. Instead, the marginal of the prior is estimated directly from the data. These parameters are still assumed to be identically and independently distributed, but the distribution family isn’t specified.

The non-parametric approach was pioneered by Herbert Robbins. The paper, Robbins [1964], is online here.

Empirical Bayes can be practical in certain circumstances, but the professor concludes by hinting at the hierarchical models we’ll learn in Unit 7. These models are some of the most interesting and widely useful in the course.