We present Monte Carlo simulations for a classical antiferromagnetic Heisenberg model with both nearest (J(1)) and next-nearest (J(2)) exchange couplings on the square lattice in the presence of nonmagnetic impurities. We show that the order-by-disorder entropy selection, associated with the Ising-like phase transition that appears for J(2)/J(1) > 1/2 in the pure spin model, is quenched at low temperature due to the presence of nonmagnetic impurities. Evidence that a new competing order is stabilized around the impurities and in turn induces a reentrance phase transition is reported. Implications for local magnetic measurement of the parent compound of iron pnictides are briefly discussed.