We present a detailed comparison of the rheology of concentrated hard and soft-sphere suspensions using a variety of techniques including large-amplitude oscillatory shear (LAOS). While the soft spheres are jammed and exhibit permanent contact, the hard-sphere suspensions are below close packing where particle collisions lead to an effective modulus. Oscillatory shear measurements are used to determine the strain-dependent viscoelastic moduli and yield stress. A recent scheme is applied to interpret LAOS data in terms of a sequence of physical processes [Rogers et al.,J. Rheol, 55, 435–458 (2011a)], revealing different characteristics of yielding, flow, and structural rejuvenation in the two systems. While for hard spheres, yielding and flow are governed by the breaking and rejuvenation of the nearest neighbor cage; for soft spheres, the particle compliance gives rise to a much more gradual yielding. We address the effect of particle softness directly by measuring the single-particle modulus with atomic force microscopy, and linking it to the suspension modulus via the pair correlation function determined by microscopy.