Speed of sound estimation in ultrasound imaging is a growing modality with several clinical applications such as hepatic steatosis stages quantification. A key challenge for clinically-relevant speed of sound estimation is to obtain repeatable values independent from superficial tissues and available in real-time. Recent works have demonstrated the feasibility to achieve quantitative estimations of the local speed of sound in layered media. However, such techniques require a high computational power and exhibit instabilities. We present a novel speed of sound estimation technique based on an angular approach of ultrasound imaging in which plane-waves are considered in transmit and in receive. This change of paradigm allows us to rely on refraction properties of plane-waves to infer local speed of sound values directly from the angular raw-data. The proposed method robustly estimates the local speed of sound with only few ultrasound emissions and with a low computational complexity which makes it compatible with real-time imaging. Simulations and in vitro experimental results show that the proposed method outperforms state-of-the-art approaches with biases and standard deviations lower than 10m/s, 8 times less emissions and 1000 times lower computational time. Further in vivo experiments validate its performance for liver imaging.