We formulate and solve a new parameter estimation problem in the presence of bounded model uncertainties. The new method is suitable when a priori bounds on the uncertain data are available, and its solution guarantees that the effect of the uncertainties will never be unnecessarily over-estimated beyond what is reasonably assumed by the a priori bounds. This is in contrast to other methods, such as total least-squares and robust estimation that do not incorporate explicit bounds on the size of the uncertainties. A geometric interpretation of the solution of the new problem is provided, along with a closed form expression for it. We also consider the case in which only selected columns of the coefficient matrix are subject to perturbations.