We investigate the persistence of micro-contacts between two elastic random rough surfaces by means of a simple model for quasi-static sliding. Contact clusters are calculated with the Boundary Element Method, then surfaces are repeatedly displaced to study the evolution of the original contact area. While the real contact area remains constant due to the rejuvenation of micro-contacts, the original contact clusters are progressively erased and replaced by new ones. We find an approximate exponential decrease of the original real contact area with a characteristic length that is influenced both by statistics of the contact cluster distribution and physical parameters. This study aims to shine light on the microscopic origins of phenomenological rate-and-state friction laws and the memory effects observed in frictional sliding.