Teleoperation in domains such as deep-sea or space often requires the completion of a set of recurrent tasks. We present a framework that uses a probabilistic approach to learn from demonstration models of manipulation tasks. We show how such a framework can be used in an underwater ROV teleoperation context to assist the operator. The learned representation can be used to resolve inconsistencies between the operator's and the robot's space in a structured manner, and as a fall-back system to perform tasks autonomously when teleoperation is not possible. We evaluate our framework with a realistic ROV task on a teleoperation mock-up with a group of volunteers, showing a significant decrease in time to complete the task when our approach is used. In addition, we illustrate how the system can execute tasks autonomously when the communication with the operator is lost.