This open access monograph presents an in-depth study of the problem of how agency fits into the physical world. In particular, the authors focus on agency as a precondition of free will. They present a detailed and physically well motivated formal model to anchor their philosophical discussion. Coverage brings together perspectives from physics, computer science, and different branches of philosophy. The book describes the agency model of Projective Simulation, its physical realisability and its quantum extensions. It situates this model within the discussion of agency in philosophy and in Artificial Intelligence. In addition, the authors highlight the role of agency in Quantum Mechanics itself, recently stressed by the Bayesian-inspired interpretation of Quantum Mechanics, QBism. They provide a comprehensive exposition of Quantum Mechanics and a reflection on the embodied nature of agents. (Quantum) indeterminism turns out to be a key resource for Projective Simulation, and for agency in general. This establishes a novel connection between agency and phenomenology. Overall, the book provides a coherent picture of agents as persisting physical entities endowed with active capacities. Such an explanation does not necessarily settle the question of the actual empirical basis of our human agency. It does, however, show that a coherent notion of agency is possible within a modern scientific world-view.