Frictional contacts lead to the formation of a surface layer called the third body, consisting of wear particles and structures resulting from their agglomerates. Its behavior and properties at the nanoscale control the macroscopic tribological performance. It is known that wear particles and surface topography evolve with time and mutually influence one another. However, the formation of the mature third body is largely uncharted territory, and the properties of its early stages are unknown. Here we show how a third body initially consisting of particles acting as roller bearings transitions into a shear-band-like state by forming adhesive bridges between the particles. Using large-scale atomistic simulations on a brittle model material, we find that this transition is controlled by the growth and increasing disorganization of the particles with increasing sliding distance. Sliding resistance and wear rate are, at first, controlled by the surface roughness, but upon agglomeration wear stagnates and friction becomes solely dependent on the real contact area in accordance with the plasticity theory of contact by Bowden and Tabor.