In this study, the hot deformation behavior of an Al-1% Mg alloy with very coarse initial grain size was investigated in terms of flow stress evolution and grain refinement mechanism. The large grain size was employed to study the traditional continuous dynamic recrystallization (CDRX) behavior below the critical strain to reach classical geometric dynamic recrystallization (GDRX), i.e., the thickness between HAGBs decreases and impingement of serrated HAGBs finally leads to the formation of new grains. The microstructure evolution during hot deformation with different deformation temperatures and strain rates on Gleeble 3800 machine was examined systematically by EBSD. It is concluded that, among the investigated conditions, the stress increases with strain and could reach steady state at small strain of ~0.01 at higher deformation temperature, while it keeps increasing at lower deformation temperature. The grain refinement mechanism clearly depends on the hot deformation condition, (micro)shear band assisted grain refinement was observed at lower deformation temperature and CDRX dominated at high deformation temperatures. In the conditions favoring CDRX, the grain refinement process is strongly grain orientation dependent, there exist some stable orientations inside which the formation of subgrain boundaries are difficult. The connection between the stress-strain behavior and microstructural evolution during hot deformation is further discussed. As compared to the traditional CDRX mechanism, the results presented in this paper provide a clearer and more complete picture of the grain refinement mechanism during hot deformation of Al-Mg alloys.