Although numerical approximation and statistical inference are traditionally covered as entirely separate subjects, they are intimately connected through the common purpose of making estimations with partial information. This book explores these connections from a game and decision theoretic perspective, showing how they constitute a pathway to developing simple and general methods for solving fundamental problems in both areas. It illustrates these interplays by addressing problems related to numerical homogenization, operator adapted wavelets, fast solvers, and Gaussian processes. This perspective reveals much of their essential anatomy and greatly facilitates advances in these areas, thereby appearing to establish a general principle for guiding the process of scientific discovery. This book is designed for graduate students, researchers, and engineers in mathematics, applied mathematics, and computer science, and particularly researchers interested in drawing on and developing this interface between approximation, inference, and learning.