The breaking of internal gravity waves in the abyssal ocean is thought to be responsible for much of the mixing necessary to close oceanic buoyancy budgets. The exact mechanism by which these waves break down into turbulence remains an active area of research and can have significant implications on the mixing efficiency. Recent evidence has suggested that both shear instabilities and convective instabilities play a significant role in the breaking of an internal gravity wave in a high Richardson number mean shear flow. We perform a systematic analysis of the stability of a configuration of an internal gravity wave superimposed on a background shear flow first considered by Howland et al. (J. Fluid Mech., vol. 921, 2021, A24), using direct-adjoint looping to find the perturbation giving maximal energy growth on this evolving flow. We find that three-dimensional, convective mechanisms produce greater energy growth than their two-dimensional counterparts. In particular, we find close agreement with the direct numerical simulations of Howland et al. (J. Fluid Mech., 2021, in press), which demonstrated a clear three-dimensional mechanism causing breakdown to turbulence. The results are shown to hold at realistic Prandtl numbers. At low mean Richardson numbers, two-dimensional, shear-driven mechanisms produce greater energy growth.