Integrating social and spatial networks will be critical to new approaches to cities as material systems of interaction. In this paper, we propose a way of doing so by focusing on the spatial and temporal conditions of formation of social networks – namely, on ‘encounters’ as a key social event. Drawing on classic approaches such as Freeman’s concept of segregation as ‘restriction on contact’ and Hägerstrand’s time-geography, and recent explorations of social media locational data, we analysed the space-time structure of potential encounters latent in the urban trajectories of agents differentiated by income levels in Rio de Janeiro, Brazil. This approach allows us to estimate agents’ urban trajectories examining geographic spatiotemporal positions in tweets, visualise income groups as potentially overlapping class networks, assess spaces of potential encounter and levels of social diversity on the streets. Finally, we discuss our findings and the utility and limitations of this approach in grasping a temporal ‘geography of potential encounters’ and segregated networks.