Performing breaks during long periods of mentally demanding activities is a simple and effective solution to briefly rest the brain and regain concentration. Furthermore, the way we use this break time may have different impacts on the task outcomes. Social assistive robots are commonly used to coach users and boost their motivation, but little is explored about their capability of guiding and supervising effective pauses focused on performance gain in cognitive tasks. This study investigated the effects of two different types of breaks, stretching/breathing exercises versus free-time, taking place in a Human-Robot Interaction of short-term memory games. The stretching was autonomously guided and supervised by a robot using machine learning methods to recognise the poses. Results showed that the type of break affects the performance differently according to the type of task, both types of breaks decreased participants' stress levels. However, the stretching/breathing intervention had a higher significant reduction in their stress and was also reported as the preferred one by the participants. No correlation between stress and performance was found.