Face recognition systems use the templates (extracted from users' face images) stored in the system's database for recognition. In a template inversion attack, the adversary gains access to the stored templates and tries to enter the system using images reconstructed from those templates. In this paper, we propose a framework to evaluate the vulnerability of face recognition systems to template inversion attacks. We build our framework upon a real-world scenario and measure the vulnerability of the system in terms of the adversary's success attack rate in entering the system using the reconstructed face images. We propose a face reconstruction network based on a new block called "enhanced deconvolution using cascaded convolution and skip connections" (shortly, DSCasConv), and train it with a multi-term loss function. We use our framework to evaluate the vulnerability of state-of-the-art face recognition models, with different network structures and loss functions (in total 31 models), on the MOBIO, LFW, and AgeDB face datasets. Our experiments show that the reconstructed face images can be used to enter the system, which threatens the system's security. Additionally, the reconstructed face images may reveal important information about each user's identity, such as race, gender, and age, and hence jeopardize the users' privacy.