Chapter 4 Markov Chain Monte Carlo
4.1 Detailed Balance & Stationary Distributions
A Markov chain transition kernel \(P(x, y)\) possesses invariant stationary distribution \(\pi(x)\) if \(\int \pi(x) P(x, y) dx = \pi(y)\).
4.1.1 Proof: Detailed Balance Implies Stationarity
Suppose \(P(x, y)\) satisfies Detailed Balance: \(\pi(x) P(x, y) = \pi(y) P(y, x)\).
Step 1 (Integrate over \(x\)):
\[\int \pi(x) P(x, y) dx = \int \pi(y) P(y, x) dx\]Step 2 (Factor out \(\pi(y)\)):
Rationale: \(\pi(y)\) is independent of integration variable \(x\), permitting factorization outside the integral:
\[\int \pi(x) P(x, y) dx = \pi(y) \int P(y, x) dx\]Step 3 (Apply Total Probability Property):
Rationale: Total probability property requires \(\int P(y, x) dx = 1\). Therefore:
\[\int \pi(x) P(x, y) dx = \pi(y) \cdot 1 = \pi(y) \quad \blacksquare\]
4.2 Derivation of Metropolis-Hastings Acceptance Ratio \(\alpha(x, y)\)
Let proposal density be \(q(y \mid x)\). Transition kernel is \(P(x, y) = q(y \mid x) \alpha(x, y)\) for \(x \neq y\).
Step 1 (Substitute into Detailed Balance):
\[\pi(x) q(y \mid x) \alpha(x, y) = \pi(y) q(x \mid y) \alpha(y, x)\]Step 2 (Rearrange Ratio):
\[\frac{\alpha(x, y)}{\alpha(y, x)} = \frac{\pi(y) q(x \mid y)}{\pi(x) q(y \mid x)}\]Step 3 (Peskun’s Selection Rule):
Setting \(\alpha(y, x) = 1\) whenever the ratio is \(< 1\) maximizes acceptance, establishing:
\[\mathbf{\alpha(x, y) = \min\left(1, \frac{\pi(y) q(x \mid y)}{\pi(x) q(y \mid x)}\right)} \quad \blacksquare\]
4.3 Derivation of Gibbs Sampler (Acceptance \(\alpha = 1\))
The Gibbs Sampler updates coordinate \(i\) drawing from full conditional \(\pi(x_i \mid x_{-i})\).
4.3.0.1 Proof of 100% Acceptance (\(\alpha = 1\)):
Candidate proposal is \(q(y \mid x) = \pi(y_i \mid x_{-i})\).
Step 1 (Evaluate MH Ratio):
\[\frac{\pi(y) q(x \mid y)}{\pi(x) q(y \mid x)} = \frac{\pi(y_i, x_{-i}) \cdot \pi(x_i \mid x_{-i})}{\pi(x_i, x_{-i}) \cdot \pi(y_i \mid x_{-i})}\]Step 2 (Apply Conditional Probability Identity):
Rationale: Utilizing \(\pi(y_i, x_{-i}) = \pi(y_i \mid x_{-i}) \pi(x_{-i})\) and \(\pi(x_i, x_{-i}) = \pi(x_i \mid x_{-i}) \pi(x_{-i})\).Step 3 (Simplification):
\[\frac{[\pi(y_i \mid x_{-i}) \pi(x_{-i})] \cdot \pi(x_i \mid x_{-i})}{[\pi(x_i \mid x_{-i}) \pi(x_{-i})] \cdot \pi(y_i \mid x_{-i})} = 1\]Conclusion: \(\alpha = \min(1, 1) = \mathbf{1}\). Every Gibbs update is accepted with probability 1. \(\blacksquare\)