G-priors

Consider this model:

\[\begin{aligned} \bm y \mid \bm\beta &\sim \MvNormal(\bm X \bm \beta, \nu\bm I) \\ \bm \beta \mid \nu & \sim \MvNormal(\bm 0, (\bm X^\prime \bm X)^{-1} \nu / \tau) \\ \bm \nu & \sim \InverseGamma(a, b) \end{aligned}\]

Then,

the conditional distribution $\bm \beta$ given $\bm y$ and $\nu$ is

\[\bm\beta \mid \bm y, \nu \sim \MvNormal(\hat{\bm\beta} / (1 + \tau), (\bm X^\prime \bm X)^{-1} \nu / (1 + \tau))\]

and the marginal distribution $\nu$ given $y$

\[\nu \mid \bm y \sim \InverseGamma\left( a + \frac{N}{2}, b + \frac{\bm y^\prime \bm y - \bm y^\prime \hat{\bm y}}{2(1 + \tau)} \right)\]

where

  • $\hat{\bm\beta} = (\bm X^\prime \bm X)^{-1} \bm X^\prime \bm y$
  • $\hat{\bm y} = \bm X \hat{\bm\beta}$
  • $N$ is the length of $\bm y$

In addition, $\nu$ can also be integrated out to obtain the evidence $p(\bm y)$ in closed form.

Note that as $\tau \rightarrow 0$, the conditional distribution of $\bm\beta$ approaches the distribution of the MLE.