Add low-rank metric adaptation #277
Description
The following paper presented a novel approach for adapting the metric:
Ben Bales, Arya Pourzanjani, Aki Vehtari, Linda Petzold, Selecting the Metric in Hamiltonian Monte Carlo, 2019. https://arxiv.org/abs/1905.11916
The idea is to adapt a low-rank (plus diagonal) approximation to the covariance matrix using a diagonal estimate and a low-rank approximation of the Hessian matrix. The main advantage of the estimated covariance being low-rank is that it can be stably estimated with many fewer draws than the dense covariance matrix. The paper also included a selection criterion for determining which rank to use.
They demonstrated in several benchmarks that the method often (but not always) outperformed diagonal and dense metric adaptation in terms of ESS/s.