Recent advancements in acquisition of three-dimensional models have been increasingly drawing attention to imaging modalities based on the plenoptic representations, such as light fields and point clouds. Since point cloud models can often contain millions of points, each including both geometric positions and associated attributes, efficient compression schemes are needed to enable transmission and storage of this type of media. In this paper, we present a detachable learning-based residual module for point cloud compression that allows for efficient scalable coding. Our proposed method is able to learn the encoding of residuals in any layered architecture, and is here implemented in a hybrid approach using both TriSoup and Octree modules from the G-PCC standard as its base layer. Results indicate that the proposed method can achieve performance gains in terms of rate-distortion when compared to both base layer alone, which is demonstrated both through objective metrics and subjective perception of quality in a rate-distortion framework. The source code of the proposed codec can be found at https://github.com/mmspg/learned-residual-pcc.