We numerically investigate the second harmonic generation from different plasmonic systems and evidence the key role played in their nonlinear response by the phase at the fundamental wavelength. In the case of a single plasmonic nanorod, the interference between the second harmonic dipolar and quadrupolar emission modes depends on their relative phase, which is deeply related to the excitation wavelength. The knowledge obtained in this simple case is then used to describe and understand the nonlinear response from a more complex structure, namely a gold nanodolmen. The complex phase evolution associated with a Fano resonance arising at the fundamental wavelength enables dramatically modifying the second harmonic emission patterns from plasmonic metamolecules within minute wavelength shifts. These results emphasize the importance of the phase in the nonlinear optical processes arising in plasmonic nanostructures, in addition to the increase in conversion yield associated with the excitation of localized surface plasmon resonances. (C) 2016 Optical Society of America