Robot Programming by Demonstration (RbD) covers methods by which a robot learns new skills through human guidance. In this work, we take the perspective that the role of the teacher is more important than just being a model of successful behaviour, and present a probabilistic framework for RbD which allows to extract incrementally the essential characteristics of a task described at a trajectory level. To demonstrate the feasibility of our approach, we present two experiments where new manipulation skills are transferred to a humanoid robot by using active teaching methods that put the human teacher in the loop of the robot's learning. The robot first observes the task performed by the user (through motion sensors) and the robot's skill is then refined by embodying the robot and putting it through the motion (kinesthetic teaching).