Adhesion between two bodies is a key parameter in wear processes. At the macroscale, strong adhesive bonds are known to lead to high wear rates, as observed in clean metal-on-metal contact. Reducing the strength of the interfacial adhesion is then desirable, and techniques such as lubrication and surface passivation are employed to this end. Still, little is known about the influence of adhesion on the microscopic processes of wear. In particular, the effects of interfacial adhesion on the wear particle size and on the surface roughness evolution are not clear and are therefore addressed here by means of molecular dynamics simulations. We show that, at short timescales, the surface morphology and not the interfacial adhesion strength dictates the minimum size of wear particles. However, at longer timescales, adhesion alters the particle motion and thus the wear rate and the surface morphology.