An adjoint method to calculate the gradient of island width in stellarators is presented and applied to a set of magnetic field configurations. The underlying method for calculation of the island width is that of Cary & Hanson (Phys. Fluids B, vol. 3, issue 4, 1991, pp. 1006-1014) (with a minor modification), and requires that the residue of the island centre be small. Therefore, the gradient of the residue is calculated in addition. Both the island width and the gradient calculations are verified using an analytical magnetic field configuration introduced by Reiman & Greenside (Comput. Phys. Commun., vol. 43, issue 1, 1986, pp. 157-167). The method is also applied to the calculation of the shape gradient of the width of a magnetic island in a National Compact Stellarator Experiment (NCSX) vacuum configuration with respect to positions on a coil. A gradient-based optimization is applied to a magnetic field configuration studied by Hanson & Cary (Phys. Fluids, vol. 27, issue 4, 1984, pp. 767-769) to minimize stochasticity by adding perturbations to a pair of helical coils. Although only vacuum magnetic fields and an analytical magnetic field model are considered in this work, the adjoint calculation of the island width gradient could also be applied to a magnetohydrodynamic (MHD) equilibrium if the derivative of the magnetic field, with respect to the equilibrium parameters, is known. Using the island width gradient calculation presented here, more general gradient-based optimization methods can be applied to design stellarators with small magnetic islands. Moreover, the sensitivity of the island size may itself be optimized to ensure that coil tolerances, with respect to island size, are kept as high as possible.