Modern smartphones are powerful platforms that have become part of the everyday life for most people. Thanks to their sensing and computing capabilities, smartphones can unobtrusively identify simple user states (e.g., location, performed activity, etc.), enabling a plethora of applications that provide insights on the lifestyle of the users. In this paper, we introduce routineSense: a system for the automatic reconstruction of complex daily routines from simple user states, implemented as an incremental processing framework. Such framework combines opportunistic sensing and user feedback to discover frequent and exceptional routines that can be used to segment and aggregate multiple user activities in a timeline. We use a comprehensive dataset containing rich geographic information to assess the feasibility and performance of routineSense, showing a near threefold improvement on the current state-of-the-art.