Particles of differing sizes are notoriously prone to segregation in shear driven flows under the action of gravity. This has important implications in many industrial processes, where particle-size segregation can lead to flow problems and reduced product quality, as well as longer product development and start-up times. Particle-size segregation also readily occurs in many hazardous geophysical mass flows (such as snow avalanches, debris flows and volcanic pyroclastic flows) and can lead to the formation of destructive bouldery flow fronts and significantly longer runouts. Although general theories exist to model particle-size segregation, the detailed functional dependence of the segregation flux on the shear rate, gravity, pressure, particle concentration, grain size and grain-size ratio is still not known. This paper describes refractive-index matched oscillatory shear-cell experiments that shed light on the segregation velocity in the two extreme cases of (i) a single large intruder rising up through a matrix of smaller grains, and (ii) a single small intruder percolating down through a matrix of large particles. Despite the sometimes markedly different time scales for segregation in these two situations, a unifying scaling law has been found that is able to collapse all the experimental data over a wide range of shear rates and grain-size ratios in the range . The resulting functional form is easily generalizable to intermediate concentrations and can quantitatively capture laboratory experiments and numerical simulations with a mix of large and small grains.