This volume is an outgrowth of a series of lectures presented at the CBMS Regional Conference held at Morehouse College, Atlanta, Georgia, on September 8-12, 1969. The purpose of these notes, which present results reported here for the first time, is two fold. First, to give a quick overview of certain aspects of the mathematical theory of automata and to prove in detail a number of deeper results in this subject. Second, to combine and utilize the various methods, chiefly the method of automata on infinite trees, to obtain a simple and transparent solution of Church's solvability problem.