Recent years have seen spectacular developments in the domain of nano-optics. Alongside the well-known techniques of super-resolution microscopy progress in nanofabrication has enabled important improvements in the fields of optical imaging and spectroscopy. Nowadays enhancements of specific near-field interactions by nano-antennae/cavities allows the optical readout of single molecule properties in a number of different systems. The observation of normal modes of vibration for example carries essential information on the orientation, link to the interface and environment in which the molecule is embedded.

This thesis develops firstly a novel theoretical frame for these vibration-cavity interactions by following the optomechanical formalism, which has proven to be an accurate treatment of a wide variety of mechanical systems interacting with a laser field. By leveraging this framework, we investigate on the one hand several quantum limits. On the other hand, for the parameter space experimentally available and for well-designed cavities, novel regimes of interaction between the collective mode of vibration of an ensemble of molecules and an optical tone seem within reach.

In parallel, this thesis experimentally explores one specific realization of such a system. In collaboration with the group of Christophe Galland we constructed the optical setups necessary to interrogate our nanostructures. The class of structures constructed, known as self-assembled nanojunctions or nanoparticles on mirror, reaches confinement of electric fields via plasmonic modes close to its fundamental limit and enables the study of many different mechanical systems whose thickness consist in a single molecule. Along this research we have developed techniques to characterize mechanical, electronic and thermal observables of nanojunctions interacting with an optical field.

Our experimental study has enabled the discovery of a novel effect related to the electronic confinement of these quasi-0-dimensional structures. The characteristic photoluminescence of metallic nanostructures is found to consistently blink in nanojunctions akin to other quantum emitters. The mechanisms explaining in detail this blinking remain debatable but the numerous experimental data collected highlight the key role of the interface in the specificities of the blinking observed. This finding opens novel ways to characterize the important and to date ill-described question of the interfaces in nanojunctions. The observed modifications of the nanojunctions optical properties might constitute an interesting step towards the development of a novel class of functional materials ('gap-materials') with enhanced and tailored electro-optical properties.

The last part of this thesis considers one technological application for this class of material by conceptually developing a new kind of detector based on the upconversion of an infrared photon to the visible; exploiting thus in a unique way the optical properties of the molecular system. The study of the quantum noises of these detectors highlights two exciting and technologically relevant directions for such devices : at room temperature these systems operate with levels of noise below state of the art devices, at lower temperatures these systems open an unexplored path towards single photon detection in a spectral range where this technology remains unavailable to date.