This thesis addresses the surprising features of zero-temperature statics and dynamics of several spin glass models, including correlations between soft spins that arise spontaneously during avalanches, and the discovery of localized states that involve the presence of two-level systems. It also presents the only detailed historiographical research on the spin glass theory. Despite the extreme simplicity of their definition, spin glasses display a wide variety of non-trivial behaviors that are not yet fully understood. In this thesis the author sheds light on some of these, focusing on both the search for phase transitions under perturbations of Hamiltonians and the zero-temperature properties and responses to external stimuli. After introducing spin glasses and useful concepts on phase transitions and numerics, the results of two massive Monte Carlo campaigns on three-dimensional systems are presented: The first of these examines the de Almeida-Thouless transition, and proposes a new finite-size scaling ansatz, which accelerates the convergence to the thermodynamic limit. The second reconstructs the phase diagram of the Heisenberg spin glass with random exchange anisotropy.