The Hall scattering factor, r, is a key quantity for establishing carrier concentration and drift mobility from Hall measurements; in experiments, it is usually assumed to be 1. In this paper, we use a combination of analytical and ab initio modeling to determine r in graphene. Although at high carrier densities r approximate to 1 in a wide temperature range, at low doping the temperature dependence of r is very strong with values as high as 4 below 300 K. These high values are due to the linear bands around the Dirac cone and the carrier scattering rates due to acoustic phonons. At higher temperatures, r can instead become as low as 0.5 due to the contribution of both holes and electrons and the role of optical phonons. Finally, we provide a simple analytical model to compute accurately r in graphene in a wide range of temperatures and carrier densities.