In recent years, the use of chatbots in education has been driven by advances in natural language processing and the increasing availability of digital education platforms. Although the added value of educational chatbots appears promising, researchers have noted that there is a need for empirical studies that explore the effects of incorporating chatbots into different learning scenarios. In this paper, we report on the integration of a rule-based chatbot into an information technology course. We conducted a controlled experiment in which half of the students were able to interact with the chatbot during Python lab sessions while the other half completed the sessions without the chatbot. Our results suggest that educational chatbots powered by short, simple, interactive scripts could have a positive impact on the user experience offered by learning technologies and could be pertinent to educators looking to integrate chatbots into their practice.