Fluorescence localization microscopy (i.e., PALM, STORM) has enabled optical imaging at nanometer-scale resolutions. The localization algorithms used in these techniques rely on fitting a 2-D Gaussian to the in-focus image of individual fluorophores. For fixed fluorophores, however, the observed diffraction pattern depends on the orientation of the underlying molecular dipole and does not necessarily correspond to a section of the system's point spread function. By using a physically realistic image formation model for dipoles to perform the fit, both the position and orientation of the dipole can be estimated with high accuracy, improving upon Gaussian localization. In this paper, we present an algorithm for joint position and orientation estimation based on a 3-D steerable filter, and show that the results are near-optimal with respect to the Cramer-Rao bounds. We show that patterns generated using estimated positions and orientations closely fit experimental measurements.