The limiting distributions of statistics used to test hypotheses about parameters on the boundary of their domains may provide very poor approximations to the finite-sample behaviour of these statistics, even for very large samples. We review theoretical work on this problem, describe hard and soft boundaries and iceberg estimators, and give examples highlighting how the limiting results greatly underestimate the probability that the parameter lies on its boundary even in very large samples. We propose and evaluate some simple remedies for this difficulty based on normal approximation for the profile score function, and then outline how higher order approximations yield excellent results in a range of hard and soft boundary examples. We use the approach to develop an accurate test for the need for a spline component in a linear mixed model.