The contribution of this paper is two-fold. First, we show how to extend the ESM algorithm to handle motion blur in 3D object track- ing. ESM is a powerful algorithm for template matching-based tracking, but it can fail under motion blur. We introduce an image formation model that explicitly considers the possibility of blur, and show it results in a generalization of the original ESM algorithm. This allows to converge faster, more accurately and more robustly even under large amount of blur. Our second contribution is an ef- ficient method for rendering the virtual objects under the estimated motion blur. It renders two images of the object under 3D perspec- tive, and warps them to create many intermediate images. By fusing these images we obtain a final image for the virtual objects blurred consistently with the captured image. Because warping is much faster that 3D rendering, we can create realistically blurred images at a very low computational cost.