We measure the cross-correlation of cosmic microwave background (CMB) lensing convergence maps derived from Atacama Cosmology Telescope data with galaxy lensing convergence maps as measured by the Canada-France-Hawaii Telescope Stripe 82 Survey. The CMB-galaxy lensing cross power spectrum is measured for the first time with a significance of 4.2 sigma, which corresponds to a 12% constraint on the amplitude of density fluctuations at redshifts similar to 0.9. With upcoming improved lensing data, this novel type of measurement will become a powerful cosmological probe, providing a precise measurement of the mass distribution at intermediate redshifts and serving as a calibrator for systematic biases in weak lensing measurements.