Aims. We present a ground-based, near-infrared search for lensed supernovae behind the massive cluster Abell 1689 at z = 0.18, which is one of the most powerful gravitational telescopes that nature provides. Methods. Our survey was based on multi-epoch J-band observations with the HAWK-I instrument on VLT, with supporting optical data from the Nordic Optical Telescope. Results. Our search resulted in the discovery of five photometrically classified, core-collapse supernovae with high redshifts of 0.671 < z < 1.703 and magnifications in the range Delta m = -0.31 to -1.58 mag, as calculated from lensing models in the literature. Owing to the power of the lensing cluster, the survey had the sensitivity to detect supernovae up to very high redshifts, z similar to 3, albeit for a limited region of space. We present a study of the core-collapse supernova rates for 0.4 < z < 2.9, and find good agreement with previous estimates and predictions from star formation history. During our survey, we also discovered two Type Ia supernovae in A 1689 cluster members, which allowed us to determine the cluster Ia rate to be 0.14(-0.09)(+0.19) SNuB h(2) (SNuB 10(-12) SNe L-circle dot,B(-1) yr(-1)), where the error bars indicate 1 sigma confidence intervals, statistical and systematic, respectively. The cluster rate normalized by the stellar mass is 0.10(-0.06)(+0.13) +/- 0.02 in SNuM h(2) (SNuM = 10(-12) SNe M-1 yr(-1)). Furthermore, we explore the optimal future survey for improving the core-collapse supernova rate measurements at z greater than or similar to 2 using gravitational telescopes, and for detections with multiply lensed images, and we find that the planned WFIRST space mission has excellent prospects. Conclusions. Massive clusters can be used as gravitational telescopes to significantly expand the survey range of supernova searches, with important implications for the study of the high-z transient Universe.