Harnessing photoexcited “hot” carriers in metallic nanostructures could define a new phase of non-equilibrium optoelectronics for photodetection and photocatalysis. Surface plasmons are considered pivotal for enabling efficient operation of hot carrier devices. Clarifying the fundamental role of plasmon excitation is therefore critical for exploiting their full potential. Here, we measure the internal quantum efficiency in photoexcited gold (Au)–gallium nitride (GaN) Schottky diodes to elucidate and quantify the distinct roles of surface plasmon excitation, hot carrier transport, and carrier injection in device performance. We show that plasmon excitation does not influence the electronic processes occurring within the hot carrier device. Instead, the metal band structure and carrier transport processes dictate the observed hot carrier photocurrent distribution. The excellent agreement with parameter-free calculations indicates that photoexcited electrons generated in ultra-thin Au nanostructures impinge ballistically on the Au–GaN interface, suggesting the possibility for hot carrier collection without substantial energy losses via thermalization.