We introduce a new model to study the effect of surface roughness on the jamming transition. By performing numerical simulations, we show that for a smooth surface, the jamming transition density and the contact number at the transition point both increase upon increasing asphericity, as for ellipsoids and spherocylinders. Conversely, for a rough surface, both quantities decrease, in quantitative agreement with the behavior of frictional particles. Furthermore, in the limit corresponding to the Coulomb friction law, the model satisfies a generalized isostaticity criterion proposed in previous studies. We introduce a counting argument that justifies this criterion and interprets it geometrically. Finally, we propose a simple theory to predict the contact number at finite friction from the knowledge of the force distribution in the infinite friction limit.