This book introduces cutting-edge methods on security in spectrum management, mobile networks and next-generation wireless networks in the era of artificial intelligence (AI) and machine learning (ML). This book includes four parts: (a) Architecture Innovations and Security in 5G Networks, (b) Security in Artificial Intelligence-enabled Intrusion Detection Systems. (c) Attack and Defense in Artificial Intelligence-enabled Wireless Systems, (d) Security in Network-enabled Applications. The first part discusses the architectural innovations and security challenges of 5G networks, highlighting novel network structures and strategies to counter vulnerabilities. The second part provides a comprehensive analysis of intrusion detection systems and the pivotal role of AI and machine learning in defense and vulnerability assessment. The third part focuses on wireless systems, where deep learning is explored to enhance wireless communication security. The final part broadens the scope, examining the applications of these emerging technologies in network-enabled fields. The advancement of AI/ML has led to new opportunities for efficient tactical communication and network systems, but also new vulnerabilities. Along this direction, innovative AI-driven solutions, such as game-theoretic frameworks and zero-trust architectures are developed to strengthen defenses against sophisticated cyber threats. Adversarial training methods are adopted to augment this security further. Simultaneously, deep learning techniques are emerging as effective tools for securing wireless communications and improving intrusion detection systems. Additionally, distributed machine learning, exemplified by federated learning, is revolutionizing security model training. Moreover, the integration of AI into network security, especially in cyber-physical systems, demands careful consideration to ensure it aligns with the dynamics of these systems. This book is valuable for academics, researchers, and students in AI/ML, network security, and related fields. It serves as a resource for those in computer networks, AI, ML, and data science, and can be used as a reference or secondary textbook.