We interact with the world through a body that includes hands and fingers. Likewise, providing an avatar allowing the control of a virtual body with fingers is an important step to improve the user experience in VR. When the user and their avatar skeleton and morphology differ, the direct application of the captured user motion on the avatar's skeleton generally induces self-contact conflicts such as undesired interpenetrations or gaps. Hence it is essential to retarget on-the-fly the user's pose to prevent such mismatches. In this paper, we propose a real-time avatar control with self-contact congruency, including finger mobility. The retargeting approach is evaluated through a subjective evaluation procedure, comparing it against a full-body animation using the user's original movement. Our results show that the retargeted animation approach outperforms the avatar control through the sole captured user movement. Avatars and Virtual Agents Mathias Delahaye, Bruno Herbelin, and Ronan Boulic