Recovery of sparse signals from linear, dimensionality reducing measurements broadly fall under two well-known formulations, named the synthesis and the analysis a ́ la Elad et al. Recently, Chandrasekaran et al. introduced a new algorithmic sparse recovery framework based on the convex geometry of linear inverse prob- lems, called the atomic norm formulation. In this paper, we prove that atomic norm formulation and synthesis formulation are equiva- lent for closed atomic sets. Hence, it is possible to use the synthesis formulation in order to obtain the so-called atomic decompositions of signals. In order to numerically observe this equivalence we derive exact linear matrix inequality representations, also known as the theta bodies, of the centrosymmertic polytopes formed from the columns of the simplex and their antipodes. We then illustrate that the atomic and synthesis recovery results agree on machine precision on randomly generated sparse recovery problems.