Using a novel noncontact technique based on optical interferometry, we quantify the nanoscale thermal fluctuations of red blood cells (RBCs) and giant unilamellar vesicles (GUVs). The measurements reveal a nonvanishing tension coefficient for RBCs, which increases as cells transition from a discocytic shape to a spherical shape. The tension coefficient measured for GUVs is, however, a factor of 4-24 smaller. By contrast, the bending moduli for cells and vesicles have similar values. This is consistent with the cytoskeleton confinement model, in which the cytoskeleton inhibits membrane fluctuations [Gov et al., Phys. Rev. Lett. 90, 228101, (2003).