Using the Rossi X-Ray Timing Explorer, we have measured similar to 27 mu s time delays in 830 Hz quasi-periodic oscillations (QPOs) between 4-6 and 11-17 keV in 4U 1608-52, with high-energy photons lagging low-energy photons, and found upper limits to the time delays of 45 mu s between 2-6.5 and 6.5-67 keV in similar to 730 Hz QPOs in 4U 0614+091 and 30 mu s between 8.7-12.4 and 12.4-67 keV in similar to 870 Hz QPOs in 4U 1636-53. We also find that the cross-coherence function between QPOs at different energies is greater than 0.85 with 95% confidence in 4U 1608-52 and 4U 1636-53. If Compton upscattering of low-energy X-rays in a region with an optical depth of a few is responsible for the delays, then the Compton upscattering region is between a few kilometers and a few tens of kilometers in size.