14. Generalized Linear Models#

In Generalized Linear Models, we are modeling \(y\) as a linear combination of \(x\) predictors, like linear regression. But, rather than assuming \(y\) is normally distributed, we can define \(y\) to be of other distributions within the exponential family. In this case, we need to use a link function.

Logistic Regression#

In logistic regression, the response is binary (or binomial). The typical link function is called logit and is defined as \(\log(\frac{p}{1-p})\). A binary response (0 or 1) is used to fit the model, and then predictions are interpreted as a probability \(p\) between 0 and 1. The equation is:

\[ \log(\frac{p}{1-p}) = \beta_0 + \beta_1 x_1 + ... + \beta_k x_k \]

In the format above, the response is in the form of log-odds (logit). Equivalently, we can take the inverse logit on both sides of the equation to have the response in terms of \(p\):

\[ p = \text{invlogit}(\beta_0 + \beta_1 x_1 + ... + \beta_k x_k )\]

where the inverse logit is \(\frac{e^p}{1+e^p}\). The Bayesian model is constructed as:

\[\begin{split} \begin{align*} y_{ij} & \sim Bern(p) && \text{likelihood}\\ \\ p & = \text{invlogit}(\beta_0 + \beta_1 x_1 + ... + \beta_k x_k) && \text{deterministic relationship}\\ \beta_j & \sim N(0,\sigma_j^2) && \text{prior: } \beta_j, \space j = 0 \text{ to } k \\ \end{align*}\end{split}\]

Other common link functions for logistic regression are probit and complementary log-log, which may be used to test different fits due to their shapes.

Possion Regression#

Poisson regression is used when the response is in the form of counts, we use a log-link function. The model is:

\[ \log(\lambda) = \beta_0 + \beta_1 x_1 + ... + \beta_k x_k \]

or equivalently:

\[ \lambda = e^{\beta_0 + \beta_1 x_1 + ... + \beta_k x_k} \]

where \(\lambda\) is the rate parameter of the Poisson distribution. The Bayesian model is:

\[\begin{split} \begin{align*} y_{ij} & \sim Poisson(\lambda) && \text{likelihood}\\ \\ \lambda & = \text{exp}(\beta_0 + \beta_1 x_1 + ... + \beta_k x_k) && \text{deterministic relationship}\\ \beta_j & \sim N(0,\sigma_j^2) && \text{prior: } \beta_j, \space j = 0 \text{ to } k \\ \end{align*}\end{split}\]