What’s your favourite MCMC method/technique atm? I know this can be quite domain dependent, but curious. From my experience, it can seem more of an art than a science at times.
Definitely Hamiltonian Monte Carlo / NUTS for the last few years. From both a practical standpoint and theoretical perspective, I see no reason why it doesn't scale to millions of parameters if you have enough GPUs. Stuff like Gibbs samplers and Metropolis-Hastings tends to run into issues once you get beyond ~100 parameters, sometimes more with clever graph structure.
I am also really interested to see if CHEES HMC (https://proceedings.mlr.press/v130/hoffman21a.html) makes it as a viable in-the-wild inference technique since it gets around the nastiness of writing NUTS as a recursive algorithm on hardware (GPU / TPU) generally not friendly to that type of control flow.
I am also really interested to see if CHEES HMC (https://proceedings.mlr.press/v130/hoffman21a.html) makes it as a viable in-the-wild inference technique since it gets around the nastiness of writing NUTS as a recursive algorithm on hardware (GPU / TPU) generally not friendly to that type of control flow.