While our daily activities usually involve interactions with others, the state-of-the-art methods on activity recognition do not exploit the relationship between social interactions and human activity. This paper addresses the problem of interpreting social activity from human-human interactions captured by mobile sensing networks. Our first goal is to discover different social activities such as chatting with friends from human-human interaction logs and then characterize them by the set of people involved, time and location of the occurring event. Our second goal is to perform automatic labeling of the discovered activities using predefined semantic labels such as coffee breaks, weekly meetings, or random discussions. Our analysis was conducted on interaction networks sensed with Bluetooth and infrared sensors by about fifty subjects who carried sociometric badges over 6 weeks. We show that the proposed system reliably recognized coffee breaks with 99% accuracy, while weekly meetings were recognized with 88% accuracy.