Gareth Roberts – Bayesian Fusion - 2020

Title : Gareth Roberts – Bayesian Fusion Author(s): MCQMC 2020 Link(s) : https://www.youtube.com/watch?v=UcWF9gadPRI

Talk about Bayesian Fusion - a method for "combining distributed Bayesian inferences from different sources into a single combined posterior summary."

This line of work started from work on Monte Carlo Fusion.

Consider the divide-and-conquer paradigm - we want to do Bayesian inference on subsets of the data (resulting in what is called sub-posteriors) and combine these. Motivations for this include problems introduced by big data, and also inference under privacy constraints where we cannot have all the data in one database.

Consider \(f(\mathbf{ x }) =\prod_{c=1}^{C}f_c(\mathbf{ x })\) to be the posterior of interest - we can draw samples from \(f_c(\mathbf{ x })\) but not \(f(\mathbf{ x })\). See Consensus Monte Carlo (Scott et al. 2016) on how to deal with such posterios when the factors are Gaussian.

Consider an auxillary variable approach - if we are interested in \(f(\mathbf{ y})=\propto \prod_{c=1}^{C}f_c(\mathbf{ y })\), consider \(g(\mathbf{ x }^{(1)},\cdots,\mathbf{ x }^{(C)},\mathbf{ y }) \propto \prod_{c=1}^{C}f_c^2(\mathbf{ x }^{(c)})p_c(\mathbf{ y }|\mathbf{ x }^{(c)})\frac{1}{f_c(\mathbf{ y })}\) where \(p_c\) is a transitional density of a Markov chain/process with stationary distribution \(f_c^2\) - the marginal distribution of \(\mathbf{ y }\) is \(f\). (Here \(f_c^2\) is squared of \(f_c\)). Many choices for \(p_c\) - here chosen to be the transition density of the double Langevin diffusion.

[TODO Fill later parts - stayfocusd stopped me midwork, should remove YouTube from blacklist].