Photons traveling cosmological distances through the inhomogeneous Universe experience a great variation in their in-medium induced effective mass. Using the EAGLE suite of hydrodynamical simulations, we infer the free electron distribution and thereby the effective photon mass after reionization. We use this data to study the inter-conversion of kinetically mixed photons and dark photons, which may occur at a great number of resonance redshifts, and obtain the "optical depth" against conversion along random lines-of-sight. Using COBE/FIRAS, Planck, and SPT measurements, we constrain the dark photon parameter space from the depletion of CMB photons into dark photons that causes both spectral distortions and additional anisotropies in the CMB. Finally, we also consider the conversion of sub-eV dark radiation into ordinary photons. We make the line-of-sight distributions of both, free electrons and dark matter, publicly available.