In this paper, we present an approach for learning a neural implicit signed distance function expressed in joint space coordinates, that efficiently computes distance-to-collisions for arbitrary robotic manipulator configurations. Computing such distances is a long standing problem in robotics as approximate representations of the robot and environment geometry can lead to overly conservative constraints, numerical instabilities and expensive computations -- limiting real-time reactive control and task success. Leveraging GPU parallelization and the differentiable nature of the proposed distance function allows for fast calculation of gradients with respect to the neural network inputs, providing a continuous repulsive vector field directly in joint space. We show that the learned high-resolution collision representation can be used to achieve real-time reactive control by i) formulating it as a collision-avoidance constraint for a quadratic programming (QP) inverse kinematics (IK), and ii) introducing it as a collision cost in a sampling-based joint space model predictive controller (MPC). For a reaching benchmark task with a 7DoF robot and dynamic obstacles intentionally obstructing the robot's path we achieve average 250Hz control frequency with QP-IK and 92Hz with MPC, showing an accelerated performance of 15% for QP-IK and 40% for MPC over baseline distance computation techniques.