In this work we present an extension of the continuous interior penalty method of Douglas and Dupont [Lecture Notes in Phys., Vol. 58 (1976) 207] to the incompressible Navier-Stokes equations. The method consists of a stabilized Galerkin formulation using equal order interpolation for pressure and velocity. To counter instabilities due to the pressure/velocity coupling, or due to high local Reynolds number, we add a stabilization term giving $L^2$-control of the jump of the gradient over element edges (faces in 3D) to the standard Galerkin formulation. Boundary conditions are imposed in a weak sense using a consistent penalty formulation due to Nitsche. We prove energy type a priori error estimates independent of the local Reynolds number and give some numerical examples recovering the theoretical results.