The richness of user-centric information gathered by modern devices can be used to keep track of memorable events, therefore acting as a prosthesis of the prone-to-forget human memory. We propose to combine virtual and physical sensors from mobile devices to infer digital memories of user activities in a semi-supervised fashion. In MemorySense, sensor data is processed by a space and energy efficient algorithm to recognize basic activities. We then use semantic reasoning to aggregate these activities into the digital equivalent of a human episodic memory.