Practical optimization problems are often hard to solve, in particular when they are black boxes and no further information about the problem is available except via function evaluations. This work introduces a collection of heuristics and algorithms for black box optimization with evolutionary algorithms in continuous solution spaces. The book gives an introduction to evolution strategies and parameter control. Heuristic extensions are presented that allow optimization in constrained, multimodal and multi-objective solution spaces. An adaptive penalty function is introduced for constrained optimization. Meta-models reduce the number of fitness and constraint function calls in expensive optimization problems. The hybridization of evolution strategies with local search allows fast optimization in solution spaces with many local optima. A selection operator based on reference lines in objective space is introduced to optimize multiple conflictive objectives. Evolutionary search is employed for learning kernel parameters of the Nadaraya-Watson estimator and a swarm-based iterative approach is presented for optimizing latent points in dimensionality reduction problems. Experiments on typical benchmark problems as well as numerous figures and diagrams illustrate the behavior of the introduced concepts and methods.