The rapid advancement of artificial intelligence (AI) has greatly influenced numerous research areas, leading to significant breakthroughs in human face-related technologies, particularly in face recognition and deepfake detection. While offering substantial benefits, these technologies pose critical challenges concerning their explainability and performance.

AI-driven face recognition technology leverages advanced algorithms to verify and identify individuals, facilitating authentication, security, and user personalization. Despite achieving remarkable accuracy, the "black box" nature of modern deep learning-based models raises significant concerns about reliability and trust. The vulnerabilities and biases of face recognition systems in more realistic situations remain underexplored. The opacity of their decision-making processes further undermines user trust and hinders the governance of this technology.

Parallel to face recognition, the advent of deepfake technologies, which manipulate or synthesize highly realistic face images and video, poses new challenges to social trust. Deepfake detection utilizes AI techniques to identify these forgeries and mitigate the adverse impact of misinformation and fraud. However, current deepfake detection methods often underperform in real-world scenarios. It is essential to understand their weaknesses in various conditions and further enhance their performance.

This thesis addresses the aforementioned challenges by proposing novel methodologies that enhance the explainability and performance of AI-driven systems in face recognition and deepfake detection.

Firstly, it approaches a better understanding of face recognition systems by examining their vulnerabilities and biases in realistic situations. We identify and categorize numerous potential influencing factors related to data quality, human characteristics, and deep learning models. An assessment framework is developed to measure the impact of these factors on modern face recognition methods. Building on these insights, advanced techniques are proposed to enhance the robustness in challenging cross-resolution and pose-variant scenarios, showing superior performance compared to the state-of-the-art.

Secondly, this thesis addresses the explainability of AI-driven face recognition systems by interpreting their decision-making process using visual saliency maps. A comprehensive explanation framework tailored for face recognition tasks is proposed. It starts by defining the use of saliency maps as explanations in both face verification and identification scenarios. A new explanation method is provided, generating precise saliency maps that adhere to this definition, followed by a quantitative evaluation methodology that facilitates performance comparison among explanation techniques. Moreover, two additional explainable face recognition methods are devised, addressing the limitations of existing explanation methods.

Lastly, this thesis advances the understanding of AI-driven deepfake detection methods. An assessment framework is proposed to evaluate the robustness of deepfake detection methods against various image and video processing operations in real-world conditions. Novel data augmentation techniques are proposed to further enhance the robustness of general deepfake detectors. Additionally, we establish a new dataset and benchmark to address open questions in the overlooked problem of detecting AI-synthetic human face images.