Hybrid quantum systems are composed of different physical components that can perform simultaneously several tasks such as quantum computing, quantum sensing and quantum communication. Photons are at the core of many of the functionalities of these systems. Photons of various wavelengths are better suited for different applications: microwave for quantum information processing, mid-infrared for sensing and visible for transmitting quantum information. The ability to study, understand and control these photons is key for this purpose. Nonlinear processes are necessary to implement some of these tasks or to connect different quantum systems. Single-crystal materials, with their unique properties, provide a promising source of nonlinearities for classical and quantum photonics. The integration of crystalline materials with optical microresonators enhances the physical effects of interest for the different applications. This thesis explores the innovative use of new materials and microresonator designs. Three main results are presented. First, a device architecture capable of direct and efficient quantum microwave-to-optical conversion. Second, a method to fabricate uncoated chalcogenide tapered fibres and to study the properties of crystalline microresonators in the mid-infrared spectral window. Third, the coupling of the excitonic emission of atomically-thin van der Waals crystals to microcavities. The findings of this thesis provide the technological basis for design and fabrication of new chipscale microresonator based devices.