Any building designed for human occupancy needs to be visually comfortable. Glare from daylight is one of the main causes of visual discomfort. Glare perception is evaluated by empirical glare models either by photometric measurements or by lighting simulations. This study explores an alternate solution that implements deep learning methods to develop glare prediction models from video recordings of human faces exposed to different levels of sunlight indoors. We trained and evaluated 12 widely used Convolutional Neural Network (CNN) architectures over a data set of 78 facial videos of 21 human participants experiencing glare in a daylit office-like setup. Results indicate that the best-performing CNN achieves an accuracy of (1) 87% in predicting glare on the repeated participants in unseen lighting conditions of different intensity and (2) 67% on new participants’ faces with previously seen lighting conditions. We propose future research directions to improve predictions from such models.