For tasks such as dressing assistance, robots should be able to adapt to different user morphologies, preferences and requirements. We propose a programming by demonstration method to efficiently learn and adapt such skills. Our method encodes sensory information (relative to the human user) and motor commands (relative to the robot actuation) as a joint distribution in a hidden semi-Markov model. The parameters of this model are learned from a set of demonstrations performed by a human. Each state of this model represents a sensorimotor pattern, whose sequencing can produce complex behaviors. This method, while remaining lightweight and simple, encodes both time-dependent and independent behaviors. It enables the sequencing of movement primitives in accordance to the current situation and user behavior. The approach is coupled with a task-parametrized model, allowing adaptation to different users’ morphologies, and with a minimal intervention controller, providing safe interaction with the user. We evaluate the approach through several simulated tasks and two different dressing scenarios with a bi-manual Baxter robot.