A Bandit Approach to Indirect Inference

Abstract: We present a novel approach to the family of parameter estimation methods known asindirect inference (II), using results from bandit optimization, a sub-field of reinforcementlearning concerned with stateless Markov decision processes (MDPs). First, we present theproblem of indirect inference and show how it may be cast into the general framework ofMDPs. We then discuss how this approach alleviates some limitations imposed by therandomness inherent to the optimization required for indirect inference. The bandit-basedapproach to indirect inference is implemented in code for two well-established banditalgorithms and subsequently validated experimentally. Our approach is demonstrated towork well in practice on the simple task of estimating the parameters of a Gaussian model,as well as on the non-trivial task of estimating the parameters of a mixture normal model.While the approach as a whole shows promise, the discretization required for analysis posescomputational problems for some types of estimation tasks: Potential avenues to refining themethod for such tasks are discussed.