Rapid advances in image sensor technology have generated a mismatch between the small size of image sensor pixels and the achievable filter spectral resolution. This mismatch has prevented the realization of chip-based image sensors with simultaneously high spatial and spectral resolution. We report here a concept that overcomes this trade-off, enabling high spectral resolution (transmission FWHM <31 nm) filters with subwavelength dimensions operating at optical and near-infrared wavelengths. An inverse design methodology was used to realize a new type of plasmonic cavity that efficiently couples an in-plane Fabry–Perot resonator to a single plasmonic slit that supports surface plasmon polaritons. This design principle, combined with a new metal imprinting method that yields metallic nanostructures with both top and bottom surfaces that are extremely smooth, enabled demonstration of high spectral resolution transmission filters with smaller area than any previously reported.