Immense field enhancement and nanoscale confinement of light are possible within nanoparticle-on-mirror (NPoM) plasmonic resonators, which enable novel optically activated physical and chemical phenomena and render these nanocavities greatly sensitive to minute structural changes, down to the atomic scale. Although a few of these structural parameters, primarily linked to the nanoparticle and the mirror morphology, have been identified, the impact of molecular assembly and organization of the spacer layer between them has often been left uncharacterized. Here, we experimentally investigate how the complex and reconfigurable nature of a thiol-based self-assembled monolayer (SAM) adsorbed on the mirror surface impacts the optical properties of the NPoMs. We fabricate NPoMs with distinct molecular organizations by controlling the incubation time of the mirror in the thiol solution. Afterward, we investigate the structural changes that occur under laser irradiation by tracking the bonding dipole plasmon mode, while also monitoring Stokes and anti-Stokes Raman scattering from the molecules as a probe of their integrity. First, we find an effective decrease in the SAM height as the laser power increases, compatible with an irreversible change of molecule orientation caused by heating. Second, we observe that the nanocavities prepared with a densely packed and more ordered monolayer of molecules are more prone to changes in their resonance compared to samples with sparser and more disordered SAMs. Our measurements indicate that molecular orientation and packing on the mirror surface play a key role in determining the stability of NPoM structures and hence highlight the under-recognized significance of SAM characterization in the development of NPoM-based applications.