Inferring the organization of fluorescently labeled nanosized structures from single molecule localization microscopy (SMLM) data, typically obscured by stochastic noise and background, remains challenging. To overcome this, we developed a method to extract high-resolution ordered features from SMLM data that requires only a low fraction of targets to be localized with high precision. First, experimentally measured localizations are analyzed to produce relative position distributions (RPDs). Next, model RPDs are constructed using hypotheses of how the molecule is organized. Finally, a statistical comparison is used to select the most likely model. This approach allows pattern recognition at sub-1% detection efficiencies for target molecules, in large and heterogeneous samples and in 2D and 3D data sets. As a proof-of-concept, we infer ultrastructure of Nup107 within the nuclear pore, DNA origami structures, and alpha-actinin-2 within the cardiomyocyte Z-disc and assess the quality of images of centrioles to improve the averaged single-particle reconstruction.