In stably stratified and rotating fluids, an axisymmetric columnar vortex can be unstable to a special instability with an azimuthal wavenumber m=1 which bends and slices the vortex into pancake vortices (Gent & McWilliams Geophys. Astrophys. Fluid Dyn., vol. 35 (1–4), 1986, pp. 209–233). This bending instability, called the ‘Gent–McWilliams instability’ herein, is distinct from the shear, centrifugal or radiative instabilities. The goals of the paper are to better understand the origin and properties of this instability and to explain why it operates only in stratified rotating fluids. Both numerical and asymptotic stability analyses of several velocity profiles have been performed for wide ranges of Froude number Frh=Ω0/N and Rossby number Ro=2Ω0/f, where Ω0 is the angular velocity on the vortex axis, N the Brunt–Väisälä frequency and f the Coriolis parameter. Numerical analyses restricted to the centrifugally stable range show that the maximum growth rate of the Gent–McWilliams instability increases with Ro and is independent of Frh for Frh⩽1. In contrast, when Frh>1, the maximum growth rate decreases dramatically with Frh. Long axial wavelength asymptotic analyses for isolated vortices prove that the Gent–McWilliams instability is due to the destabilization of the long-wavelength bending mode by a critical layer at the radius rc where the angular velocity Ω is equal to the frequency ω: Ω(rc)=ω. A necessary and sufficient instability condition valid for long wavelengths, finite Rossby number and Frh⩽1 is that the derivative of the vertical vorticity of the basic vortex is positive at rc: ζ′(rc)>0. Such a critical layer rc exists for finite Rossby and Froude numbers because the real part of the frequency of the long-wavelength bending mode is positive instead of being negative as in a homogeneous non-rotating fluid (Ro=Frh=∞). When Frh>1, the instability condition ζ′(rc)>0 is necessary but not sufficient because the destabilizing effect of the critical layer rc is strongly reduced by a second stabilizing critical layer rc2 existing at the radius where the angular velocity is equal to the Brunt–Väisälä frequency. For non-isolated vortices, numerical results show that only finite axial wavenumbers are unstable to the Gent–McWilliams instability.