Purpose: In this work, we present a new machine learning method based on the transformer neural network to detect eye rubbing using a smartwatch in a real-life setting. In ophthalmology, the accurate detection and prevention of eye rubbing could reduce incidence and progression of ectasic disorders, such as keratoconus, and to prevent blindness. Methods: Our approach leverages the state-of-the-art capabilities of the transformer network, widely recognized for its success in the field of natural language processing (NLP). We evaluate our method against several baselines using a newly collected dataset, which consist of data from smartwatch sensors associated with various hand-face interactions. Results: The current algorithm achieves an eye-rubbing detection accuracy greater than 80% with minimal (20 minutes) and up to 97% with moderate (3 hours) user-specific fine-tuning. Conclusions: This research contributes to advancing eye-rubbing detection and establishes the groundwork for further studies in hand-face interactions monitoring using smartwatches. Translational Relevance: This experiment is a proof-of-concept that eye-rubbing detection is effectively detectable and distinguishable from other similar hand gestures, solely through a wrist-worn device and could lead to further studies and patient education in keratoconus management.