Using Approximative Methods in Riemann Manifold Metropolis Adjusted Langevin Algorithm

Abstract: When sampling from target densities with high correlation using the Metropolis-Hastings algorithm, performance can be improved by considering a discretization of a Langevin diffusion defined on the Riemann manifold as proposal mechanism. Through the metric tensor of the manifold, it is possible to capture the curvature of the parameter space and adjust for the correlation and differences in magnitude of variance. However, these methods need analytical expressions of several quantities, such as the expected Fisher information matrix and its derivative, which may be cumbersome to derive in more challenging problems. This calls for approximative methods. In this bachelor thesis, two approximative methods are investigated. The first method uses second-order finite differences to approximate the metric tensor and the second method uses the gradient of the log posterior to approximate the expected Fisher information matrix and through this the metric tensor. The methods are assessed by performing inference on normal distributed and gamma distributed data and the results are compared to the results of an analytical implementation, a Metropolis adjusted Langevin algorithm not using the Riemann manifold and a Random Walk Metropolis-Hastings algorithm. The approximative methods turn out to be computationally slower than the analytical implementation, but the time normalized effective sampling size is of the same magnitude as for the analytical method, and the same improvement compared to the Metropolis adjusted Langevin algorithm and the Random Walk Metropolis-Hastings algorithm is observed in both the analytical and the approximative methods.