Integrating social and spatial networks will be critical to new approaches to cities as systems of interaction. In this paper, we focus on the spatial and temporal conditions of encounters as a key condition for the formation of social networks. Drawing on classic approaches such as Freeman’s concept of segregation as ‘restriction on contact’, Hägerstrand’s time-geography, and recent explorations of social media locational data, we analysed the space-time structure of potential encounters latent in the urban trajectories of people with different income levels in Rio de Janeiro, Brazil. This approach allows us to estimate trajectories examining spatiotemporal positions in tweets, and assess spaces of potential encounter and levels of social diversity on the streets. Finally, we discuss the utility and limitations of an approach developed to grasp how clusters of encounters between groups with different income levels are produced.