Optical detection back-action in cantilever resonant or static detection presents a challenge when striving for state-of-the-art performance. The origin and possible routes for minimizing optical back-action have received little attention in literature. Here, we investigate the position and mode dependent optical back-action on cantilever beam resonators. A high power heating laser (100 mu W) is scanned across a silicon nitride cantilever while its effect on the first three resonance modes is detected via a low-power readout laser (1 mu W) positioned at the cantilever tip. We find that the measured effect of back-action is not only dependent on position but also the shape of the resonance mode. Relevant silicon nitride material parameters are extracted by fitting finite element (FE) simulations to the temperature-dependent frequency response of the first three modes. In a second round of simulations, using the extracted parameters, we successfully fit the FEM results with the measured mode and position dependent back-action. From the simulations, we can conclude that the observed frequency tuning is due to temperature induced changes in stress. Effects of changes in material properties and dimensions are negligible. Finally, different routes for minimizing the effect of this optical detection back-action are described, allowing further improvements of cantilever-based sensing in general.