5 Feb, 2016

Posterior Consistency

$\Phi_n \rightarrow 0$ under $\beta^0$ as $n\rightarrow\infty$.

\[\begin{aligned} E_{\beta^0} (\Phi_n) &\le c_1 \exp(-cn) \\\\ \underset{\beta\in B_\epsilon^C}{Sup}\brak{ E_{\beta} (1-\Phi_n)} &\le c_2 \exp(-bn) \\\\ \end{aligned}\]

where the constants are positive.

Next Section

  1. Sequential MCMC
  2. Assumed Density Filtering (C-DF)
  3. Sequential Monte Carlo (SMC)
  4. MCMC for big $n$ with Stochastic Gradient Descent
  5. GP for big $n$

Sequential MCMC (2016 Annals of Statistics)

Suppose you have big data or you are dealing with streaming data (read data as they come, no storage bottleneck - does not require storage).

  • Circumstances:
    1. Not allowed to store entire data.
    2. Data coming in batches. It is not desireable to store data until every time point. That is if you are at time $t$, you are not allowed to use the full data until time point $t$.
    3. At time $t$, you will only use the full $D_t$. But you only use summary measures from $D_1,…D_{t-1}$
  • At every time point, you will update and propogate those summary measures.

Algorithm for SMC

  1. Assume that you have decided to fit a model to the data and the model involves parameters $\theta$
  2. Say $\pmb\theta$ is of dimension $K$, and $\theta_i|\theta_{-i},y_t$ has a recognizable form that can be easily sampled from. ($y_t = \paren{D_1,…,D_t}$)
  3. Assume $\theta_i | \theta_{-i},y_t$ is dependent on $y_t$ only through a summary statistic.

Homework Tip

  • Gelman BDA3 (p.377) for Spike and Slab