The ability to predict pedestrian behaviour is crucial for road safety, traffic management systems, Advanced Driver Assistance Systems (ADAS), and more broadly autonomous vehicles. We present a vision-based system that simultaneously locates where pedestrians are in the scene, estimates their body pose and predicts their intention to cross the road. Given a single image, our proposed neural network is designed using a bottom-up approach and thus runs at nearly constant time without relying on a pedestrian detector. Our method jointly detects human body poses and predicts their intention in a multitask framework. Experimental results show that the proposed model outperforms the precision scores of the state-of-the-art for the task of intention prediction by approximately 20% while running in real-time (5 fps). The source code is publicly available so that it can be easily integrated into an ADAS or into any traffic light management systems.