Roombots (RB) are self-reconfigurable modular robots designed to explore physical structure change by robotic reconfiguration and adaptive locomotion on structured grid environment or unstructured environment. The primary goal of RB is to create adaptive furniture. In this study, we propose a novel and user-friendly interface to control position and intensity of a mobile desk light using RB modules. In the proposed method, the user interacts with the RB with only hand/arm gestures. The user’s arm is tracked with a single Kinect having bird’s eye view. We demonstrate the effectiveness of the proposed interface in real hardware setup and discuss contributions of it.