Markov Chain Monte Carlo (MCMC)

An iterative approach to generate samples from some target distribution \(\pi\) defined on a state space \(\mathcal{X}\) by constructing a Markov Chain which has \(\pi\) as its stationary distribution.

MCMC is often used in Bayesian modelling where we want to sample from the posterior distribution, which is often known up to a normalizing constant.

The time it takes to reach the stationary distribution \(\pi\) is called the mixing time. Since initial samples are not from \(\pi\) they are thrown away.