Since the invention of first camera, people have always thought of new ways to capture moments. With the advent of digital photography, not only imaging systems have progressed, but also technology provides new user experience. The recently introduced light field photography enables users with new imaging modalities such as digital refocusing and looking through occlusions, for example. The specific structure of light field signals highlights the importance of adapting different routines of traditional imaging to exploit the power of light field photography. In recent years, the consumer market has expressed a huge interest in analyze and editing of light field images which necessitates adaptation of traditional background subtraction methods, as the first step of most editing tools, to operate in the light field space. The goal of this work is to develop a computationally efficient algorithm making use of the properties of light fields for background subtraction. Our approach is based on a novel transform, which maps each light ray into a new space where background subtraction is achieved by a simple thresholding.