Luminous broad H alpha emission and red rest-optical spectral energy distributions (SEDs) are the hallmark of compact little red dots (LRDs), implying highly attenuated dusty starbursts and/or obscured active galactic nuclei (AGN). However, the lack of observed far-infrared (FIR) emission has proved difficult to reconcile with the implied attenuated luminosity in these models. Here, we utilize deep new Atacama Large Millimeter/submillimeter Array imaging, new and existing JWST/MIRI imaging, and archival Spitzer/Herschel imaging of two of the rest-optically brightest LRDs (z = 3.1 and z = 4.47) to place the strongest constraints on the IR luminosity in LRDs to date. The detections at lambda rest = 1-4 mu m imply flat slopes in the rest-IR, ruling out a contribution from hot (T greater than or similar to 500 K) dust. Similarly, FIR nondetections rule out any appreciable cold (T less than or similar to 75 K) dust component. Assuming energy balance, these observations are inconsistent with the typical FIR dust emission of dusty starbursts and quasar tori, which usually show a mixture of cold and hot dust. Additionally, our [C ii] nondetections rule out typical dusty starbursts. We compute empirical maximum IR SEDs and find that both LRDs must have log(LIR/L circle dot)less than or similar to 12.2 at the 3 sigma level. These limits are in tension with the predictions of rest-optical spectrophotometric fits, be they galaxy-only, AGN-only, or composite. It is unlikely that LRDs are highly dust-reddened intrinsically blue sources with a dust temperature distribution that conspires to avoid current observing facilities. Rather, we favor an intrinsically redder LRD SED model that alleviates the need for strong dust attenuation.